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This  dissertation  has  provided  additional  evidence  to  facili- 
tate a  better  understanding  of  the  role  of  auditor  judgment  in  the 
evaluation  of  IAC  information  and  the  decision  as  to  the  extent  of 
subsequent  audit  procedures  (substantive  tests).   A  laboratory  experi- 
ment was  conducted  to  gather  data  which  were  used  to  examine  auditor 
judgment  about  the  reliability  of  an  IAC  system  and  the  resulting 
selection  of  a  sample  size  for  a  substantive  test.   The  participants 
were  109  members  of  four  "Big-Eight"  accounting  firms.   A  descriptive 
approach  was  taken  within  the  framework  provided  by  the  Brunswik  lens 
model  to  examine  these  decisions  for  a  specific  audit  task. 

Two  key  decision  points  within  the  audit  process  that  require 
the  exercise  of  professional  judgment  were  examined  in  this  study. 
These  included:   (1)  the  preliminary  audit  planning  stage  and  (2)  the 


audit  program  revision  stage.   Examination  of  the  responses  from  these 
stages  resulted  in  descriptions  of  the  auditors'  judgments  and  the 
evaluation  of  auditor  consensus  and  self-insight. 

Responses  of  the  auditors  at  both  the  preliminary  audit  stage 
and  at  the  audit  program  revision  stage  suggest  that,  within  the  con- 
text of  this  study,  the  most  important  of  three  key  controls  in  decid- 
ing that  "recorded  sales  are  for  valid  transactions"  is  the  "control 
over  physical  shipment."  The  controls  for  "shipment  authorization"  and 
"credit  approval"  followed  in  importance  in  that  order. 

The  extent  of  judgment  consensus  at  the  audit  program  revision 
stage,  as  examined  through  the  application  of  canonical  correlation  and 
Pearson  product-moment  correlation,  was  found  to  be  slightly  lower  than 
in  other  audit  studies.   The  complexity  of  the  audit  task  was  thought 
to  be  a  contributing  factor  to  this  finding.    The  nature  of  the  audit 
task  also  was  thought  to  contribute  to  the  relatively  low  self-insight 
indices  that  were  found.   Firm  differences  and  experience  levels  were 
found  to  be  significant  in  examining  the  importance  of  controls,  the 
extent  of  consensus,  and  the  level  of  self-insight  exhibited  by  the 
auditors. 

In  summary,  although  other  studies  have  examined  similar  deci- 
sions made  by  auditors,  some  specific  and  significant  limitations  were 
thought  to  exist  in  these  studies.   The  present  study  sought  to  over- 
come these  limitations.   Primarily,  a  more  realistic  setting  was  used, 
with  a  greater  degree  of  control  over  the  significant  factors  impacting 
upon  the  decisions  of  the  auditors.   This  research  has  been  presented 
with  the  hope  that  it  can  in  some  way  serve  as  a  stepping  stone  for 
future  research  in  this  vitally  important  area. 


CHAPTER  I 
INTRODUCTION 

The  demand  for  the  services  provided  by  external  auditors  has 
increased  in  recent  years  as  the  complexities  of  the  business  world 
have  increased.   The  knowledge  and  expertise  of  the  independent  auditor 
have  resulted  in  a  natural  dependency  on  the  auditor  to  provide  infor- 
mation or  services  desired  by  management,  interested  third  parties,  or 
the  general  public. 

As  prescribed  by  the  second  standard  of  field  work  [AICPA, 
1979a,  Section  320],  a  primary  area  of  interest  and  concern  of  the 
auditor  when  engaged  to  perform  an  audit  is  the  functioning  of  the 
internal  accounting  controls  (IACs).   Internal  control  has  been  defined 
as  follows: 

Internal  control  comprises  the  plan  of  organization 
and  all  of  the  coordinate  methods  and  measures  adopted 
within  a  business  to  safeguard  its  assets,  check  the 
accuracy  and  reliability  of  its  accounting  data,  pro- 
mote operational  efficiency,  and  encourage  adherence 
to  prescribed  managerial  policies  [AICPA,  1979a, 
Section  320.09]. 

This  definition  incorporates  both  accounting  and  administrative 
controls.    However,  the  extent  of  the  auditor's  concern  for  internal 
control  depends  primarily  upon  the  type  of  engagement  or  audit  objective. 
Recent  reports  relating  to  the  evaluation  of  internal  control  (i.e., 


See  AICPA  Professional  Standards,  Sections  320.09-320.13 
for  a  discussion  of  administrative  versus  accounting  controls. 


The  Foreign  Corrupt  Practices  Act  of  1977,  Required  Communication  of 
Material  Weaknesses  in  Internal  Accounting  Control  [AICPA,  1979a, 
Section  323],  and  the  Report  of  the  Special  Advisory  Committee  on 
Internal  Accounting  Control  [AICPA,  1979b])  have  publicized  the  signi- 
ficance of  IAC  evaluation  to  both  management  and  external  auditors. 
The  primary  importance  of  IAC  evaluation  from  the  external 
auditor's  perspective  lies  in  the  impact  of  the  evaluation  of  the  audit 
tests  to  be  performed.   As  indicated  in  the  AICPA  Professional  Stan- 
dards: 

There  is  to  be  a  proper  study  and  evaluation  of  the 
existing  internal  control  as  a  basis  for  reliance 
thereon  and  for  the  determination  of  the  resultant 
extent  of  the  tests  to  which  auditing  procedures 
are  to  be  restricted  [AICPA,  1979a,  Section  320.01] 
(underlining  added). 

The  present  study  examines  this  particular  relationship  within  a 
specific  audit  setting. 

This  chapter  introduces  the  concepts  of  IAC  evaluation,  com- 
pliance testing  and  substantive  testing,  followed  by  brief  discussions 
of  the  relationships  among  these  concepts  and  the  importance  of  audit 
judgment  to  the  audit  process.   The  research  objectives  of  the  study 
are  presented  and  discussed;  a  brief  description  of  the  research 
approach  then  is  provided.   Finally,  the  organization  of  the  disserta- 
tion is  presented. 

Compliance  and  Substantive  Tests 

The  extent  of  reliance  on  IAC  is  determined  by  evaluation  of 
the  results  of  compliance  testing.   Compliance  testing  is  used  to 
provide  reasonable  assurance  that  the  control  procedures  within  a 


company  are  operating  as  prescribed  through  preliminary  evaluation. 
The  results  of  compliance  testing  provide  the  necessary  information 
to  determine  the  nature,  timing,  and  extent  of  substantive  procedures. 

Substantive  testing  includes  those  procedures  necessary  to 
obtain  sufficient  competent  evidential  matter  as  prescribed  in  the 
AICPA's  third  standard  of  field  work  [AICPA,  1979a,  Section  330]. 
These  procedures  include:   (a)  tests  of  details  of  transactions  and 
balances  and  (b)  analytical  review  procedures  applied  to  financial 
information.   Therefore,  it  is  a  combination  of  compliance  test  pro- 
cedures and  substantive  test  procedures  that  constitutes  the  main 
component  of  the  audit  program  that  is  to  be  followed. 

Auditor  Judgment 

As  a  result  of  the  additional  emphasis  on  generating  LAC  infor- 
mation, we  might  expect  to  find  a  redistribution  of  resources  by  the 
auditor  with  regard  to  the  appropriate  combination  of  audit  procedures, 
or  higher  audit  fees  for  the  additional  work  that  would  be  required. 
From  a  research  perspective,  there  is  an  obvious  need  for  additional 
examination  of  the  auditor's  decisions  that  result  from  the  exercise 
of  his  professional  judgment.   Specifically,  his  incorporation  of  the 
important  inputs  of  preliminary  IAC  evaluation  and  compliance  test 
results  into  a  decision  regarding  the  optimal  extent  of  additional 
audit  procedures  (substantive  tests)  warrants  examination. 

Professional  judgment  is  fundamental  to  the  auditing  profes- 
sion and  pervades  the  entire  audit  process.   Many  statements  have 
been  made  regarding  the  necessity  to  exercise  "judgment"  in  evaluat- 
ing both  the  qualitative  and  quantitative  considerations  in  an  audit 


situation.   Some  statements  found  in  the  AICPA  Professional  Standards 
that  relate  specifically  to  this  study  include  the  following; 

In  the  observance  of  generally  accepted  auditing 
standards,  the  independent  auditor  must  exercise  his 
judgment  in  determining  which  auditing  procedures  are 
necessary  in  the  circumstances  to  afford  a  reasonable 
basis  for  his  opinion.   His  judgment  is  required  to 
be  the  informed  judgment  of  a  qualified  professional 
person  [AICPA,  1979a,  Section  110.04]  (underlining 
added) . 

...the  purpose  of  tests  of  compliance  with  accounting 
control  procedures  is  to  provide  "a  reasonable  degree 
of  assurance  that  they  are  in  use  and  are  operating  as 
planned."   What  constitutes  a  "reasonable"  degree  of 
assurance  is  a  matter  of  auditing  judgment , . . .  [AICPA, 
1979a,  Section  320.60]  (underlining  added). 

Although  statistical  sampling  furnishes  the  auditor 
with  a  measure  of  precision  and  reliability,  statis- 
tical techniques  do  not  define  for  the  auditor  the 
values  of  each  required  to  provide  audit  satisfaction. 
Specification  of  the  precision  and  reliability  neces- 
sary in  a  given  test  is  an  auditing  function  and  must 
be  based  upon  judgment  in  the  same  way  as  is  the 
decision  as  to  audit  satisfaction  required  when 
statistical  sampling  is  not  used  [AICPA,  1979a,  Sec- 
tion 320A.03]  (underlining  added). 

The  amount  and  kinds  of  evidential  matter  required  to 
support  an  informed  opinion  are  matters  for  the  auditor 
to  determine  in  the  exercise  of  his  professional  judg- 
ment  after  a  careful  study  of  the  circumstances  in  the 
particular  case  [AICPA,  1979a,  Section  330.09]  (under- 
lining added) . 

The  fact  that  professional  judgment  plays  a  major  role  in  the 
decision  process  of  auditors  has  been  recognized  for  many  years.   Mautz 
[1959,  p.  44]  presented  conclusions  concerning  auditor  judgment  as 
follows:   (1)  judgment  must  play  a  major  role  in  auditing,  and  (2) 
auditors  would  do  well  to  recognize  this  and  acquaint  themselves  with 
the  process  of  judgment  formation. 


Although  the  following  statement  was  made  in  reference  to 
accounting  in  general,  it  is  particularly  germane  to  the  auditing  proc- 
ess and  the  specific  purpose  of  this  study: 

Judgment  is,  of  course,  a  vital  part  of  any  profes- 
sional's work.   In  accounting  it  plays  an  important 
role  every  step  of  the  way.   But  that  does  not  mean 
that  it  is  a  mysterious  process,  undefinable  and 
inexplicable.   We  know  that  the  processes  that  feed 
professional  judgment  are  varied  and  complex,  yet  it 
does  not  follow  that  we  cannot  make  some  progress  in 
their  analysis  and  description  [Bernstein,  1967,  p.  9]. 

Recently,  there  has  been  increased  recognition  that  auditor  judgment 
must  be  further  evaluated  and  its  impact  understood.   A  number  of 
recent  research  publications,  e.g.,  Kennedy  [1977],  Gibbins  [1977], 
and  the  Journal  of  Accounting  Research  Supplement  on  Human  Information 
Processing  [1976] ,  have  presented  discussions  of  recent  studies  that 
concern  auditor  judgment.   This  study  also  emphasizes  the  importance 
of  auditor  judgment  as  recognized  in  the  specific  research  objectives 
discussed  in  the  following  section. 

Research  Objectives 

The  objective  of  this  study  is  to  examine  and  describe  (within 
a  specific  audit  setting)  the  impact  of  internal  control  information 
(including  the  specific  results  of  compliance  tests)  on  the  auditor's 
judgment  concerning  the  reliability  of  the  IAC  system  and  on  his 
resulting  decision  as  to  the  extent  of  substantive  testing.   This 
objective  is  pursued  through  the  use  of  a  laboratory  experiment  which 
utilizes  auditors  from  four  of  the  "Big-Eight"  public  accounting  firms. 
Questions  relevant  to  this  particular  objective  and  addressed  in  this 
study  can  be  summarized  as  follows: 


(1)  To  what  extent  do  auditors  reflect  consensus  in  terms 
of  their  specification  of  the  reliability  of  a  client's 
internal  controls  (both  prior  to  and  following  the 
evaluation  of  compliance  tests)  and  to  what  extent  is 
there  consensus  at  the  later  point  in  the  audit  process 
where  the  question  as  to  the  appropriate  amount  of  sub- 
stantive testing  to  perform  is  addressed? 

(2)  How  do  auditors  respond  to  the  results  of  compliance 
testing  in  terms  of  deciding  the  appropriate  amount  of 
substantive  testing,  when  they  are  presented  with  results 
that  may  confirm  or  disconfirm  their  original  beliefs 
about  the  existence  and  effectiveness  of  key  controls? 

(3)  How  important  are  the  various  key  internal  controls  in 
arriving  at  the  appropriate  amount  of  substantive  testing 
as  determined  by:   1)  the  confidence  levels  chosen  for 
compliance  testing,  2)  the  subjective  weights  assigned  by 
auditors,  and  3)  weights  determined  from  the  analysis-of- 
variance  (ANOVA)  technique?   And,  to  what  extent  do 
auditors  exhibit  self-insight  in  understanding  the  rela- 
tive importance  of  the  key  internal  controls? 

(4)  Do  such  factors  as  firm  differences  and  experience  levels 
have  a  significant  effect  on  the  decisions  made  by 
auditors? 


Judgment  Consensus 

Consensus  is  reflected  by  agreement  among  auditors  when  using 
the  same  data  to  reach  a  decision.   Lack  of  consensus,  or  disagreement 
among  auditors,  is  thought  by  many  to  be  costly.   For  example,  Joyce 
has  stated  this  concern  as  follows: 

Within  firms,  the  existence  of  the  review  process  sug- 
gests that  individual  differences  exist  and  are  likely 
to  be  costly  if  unresolved.   The  increasing  concern 
within  the  profession  about  "quality  control"  issues  .  .  . 
The  existence  of  continuing  education  programs  within 
at  least  the  major  audit  firms  is  further  evidence  of 
their  willingness  to  consume  resources  to  restrict 
judgment  variance  among  their  professional  staff  [1976, 
P-  31]. 

This  study  is  expected  to  provide  additional  evidence  as  to 
the  extent  of  agreement  among  auditors  when  provided  with  a  particular 
audit  situation.   Although  the  purpose  of  this  study  is  not  to  provide 
any  information  as  to  the  costs  of  any  differences  that  may  be  found, 
additional  information  as  to  the  source  of  such  differences  would  be  a 
meaningful  contribution.   The  study  evaluates  decisions  made  by  the 
auditors  at  both  a  preliminary  decision  point  in  the  audit  process 
(prior  to  conducting  compliance  tests)  and,  more  extensively,  at  the 


audit  program  revision  stage  (after  evaluation  of  compliance  test 

2 
results).    This  analysis  may  provide  insight 

disagreement  among  auditors  begins  to  emerge. 


2 
results).    This  analysis  may  provide  insights  into  the  point  at  which 


2 
See  Chapter  II  above  for  a  discussion  of  these  various  stages 
of  the  audit  process. 


Evaluation  of  Evidence 

The  independent  auditor's  objective  is  to  obtain  sufficient 
competent  evidential  matter  to  provide  him  with  a  reasonable  basis  for 
forming  an  opinion  under  the  circumstances.   He  is  also  expected  to  be 
thorough  in  his  search  for  evidential  matter  and  objective  in  its 
evaluation  [AICPA,  1979a,  Section  330 .09-330.15] .   However,  very  little 
is  known  concerning  the  auditor's  reaction  to,  and  incorporation  of, 
the  evidence  he  has  gathered. 

Of  primary  interest  in  this  study  is  the  auditor's  reaction  to 
evidence  provided  by  TAC  information.   The  results  of  this  study  should 
provide  insights  into  the  auditor's  evaluation  of  information  that  has 
been  lacking  in  previous  studies.   This  opportunity  is  enhanced  by  the 
requirement  that  the  auditor  use  a  statistical  sampling  approach  in 
evaluating  tne  evidence  concerning  the  key  IACs  within  a  controlled 
audit  environment. 

Importance  of  Controls  and  Self-Insight 

The  fact  that  the  auditors  will  be  presented  with  a  specific 
audit  objective  and  the  key  controls  to  be  considered  in  meeting  the 
objective  provide  an  excellent  opportunity  to  evaluate  the  relative 
importance  of  the  controls.   There  are  three  possible  methods  for 
determining  the  relative  importance  of  the  controls.   First,  confi- 
dence levels  used  in  determining  sample  sizes  for  the  compliance  tests 
can  be  used  to  infer  the  relative  importance  of  the  three  IACs. 
Second,  the  ANOVA  technique  provides  a  measure  of  importance  from  the 
evaluation  of  the  decisions  made  by  the  auditors  after  their  evaluation 


of  the  compliance  test  results.   Finally,  a  post- experiment  question 
requires  a  subjective  weighting  of  the  relative  importance  of  the 
three  key  controls. 

The  indications  of  the  relative  importance  of  the  controls 
allow  for  additional  evaluation  of  a  measure  of  consensus  and  also  for 
a  calculation  of  a  self-insight  index.   This  ability  of  the  auditor  to 
estimate  the  relative  importance  he  places  on  the  cues  (controls)  in 
making  his  judgments  is  considered  most  important  within  the  auditing 
profession,   Joyce,  in  discussing  the  importance  of  self-insight  to 
auditors,  made  the  following  statement: 

One  of  the  implications  of  poor  self-insight  in  situ- 
ations where  a  considerable  amount  of  professional 
expertise  is  communicated  verbally  "on  the  job" 
(between  senior  and  junior  accountants  on  an  audit, 
for  example)  is  clear:   A  distorted  representation 
of  one  professional's  decision  behavior  will  be 
transmitted  to  another  professional  [1976,  p.  52]. 

This  study  will  provide  evidence  that  may  aid  the  assessment 
of  the  extent  to  which  the  lack  of  self- insight  represents  a  signifi- 
cant problem  within  the  auditing  profession. 

Firm  and  Experience  Effects 

This  study  will  also  provide  information  as  to  the  effect  of 
two  key  factors  on  the  judgments  made  by  the  auditors.   Of  specific 
interest  are  differences  that  may  be  attributable  to  differences  in 
the  firm  affiliation  of  auditors  and/or  differences  between  various 
experience  levels. 
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Research  Approach 

Past  research  concerning  the  audit  process  has  been  both  of  a 
normative  and  a  descriptive  nature.   This  study  will  take  a  descriptive 
approach.   Support  for  this  type  of  research  is  provided  by  Kaplan: 

It  seems  difficult  to  make  a  central  attack  on  the  audit 
process  because  so  little  is  known  about  what  actually 
constitutes  a  good  audit.   This  suggests  that,  rather 
than  start  with  research  on  normative  models  in  an 
attempt  to  improve  existing  audits,  we  devote  signifi- 
cant resources  to  developing  descriptive  models  of  what 
auditors  are  actually  doing  now  [1977,  p.  9]. 

Further  support  for  this  approach  is  provided  by  recent  calls  for  addi- 
tional research  efforts  in  the  area  of  audit  judgment  by  Kaplan  [1977] 
and  Libby  and  Lewis  [1977]. 

Recently,  accounting  researchers  have  adapted  the  "lens  model" 
approach  from  psychology  to  the  examination  of  judgments  made  in 
accounting.   This  particular  approach  appears  most  consistent  with  the 
descriptive  nature  of  this  study.   The  appropriateness  of  the  lens 
model  for  a  study  such  as  this  has  been  recognized  by  Libby  and  Lewis: 

.  .  .  this  approach  (lens  model)  is  particularly  useful 
in  studying  the  impact  of  information  set  variables  on 
decision  rule  form,  stability  or  learning,  cue  usage, 
and  decision  accuracy,  reliability,  and  predictability 
or  in  descriptive  studies  of  these  variables.   This 
method  of  modeling  judgment  provides  a  compromise 
between  the  overly  simplified  approach  of  asking  sub- 
jects to  describe  the  weights  they  place  on  information 
and  the  more  complex  and  expensive  process  tracing 
models  that  have  been  used  in  the  study  of  judgment 
[1977,  p.  248]  (underlining  added). 

A  more  extensive  discussion  of  the  "lens  model"  methodology  is 
presented  in  Chapter  III. 
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The  "lens  model"  will  be  used  in  this  study  to  examine  the 
auditors'  decisions  with  respect  to  the  evaluation  of  internal  control 
information.   More  specifically,  the  audit  program  revision  stage  of 
the  experiment  is  arranged  in  a  2x3x2  factorial  design  which  results 
in  twelve  case  situations  presented  to  each  auditor.   The  analysis  of 
the  responses  is  discussed  in  Chapter  III.   The  contents  of  the  remain- 
ing chapters  are  summarized  below. 

Dissertation  Organization 

This  dissertation  is  composed  of  five  chapters.   Chapter  II 
presents  a  more  detailed  discussion  of  the  audit  process  and  the 
roles  of  statistical  sampling  and  professional  judgment  at  various 
stages  of  the  process.   A  better  understanding  and  description  of  the 
audit  process  enables  a  focus  on  the  key  decision  points  which  are  of 
interest  in  this  study.   Also,  the  roles  of  statistical  sampling  and 
professional  judgment  are  evaluated  and  the  complementary  nature  of 
their  relationship  is  established.   Relevant  literature  reviews  for 
these  areas  are  presented  and  discussed  throughout  Chapter  II. 

The  general  research  design  and  methodologies  used  are  described 
in  Chapter  III  along  with  the  details  of  the  experimental  setting  and 
the  materials  utilized.   Characteristics  of  this  study  representing 
both  strengths  and  limitations  are  also  noted  in  this  chapter. 

Chapter  IV  presents  the  results  of  the  study  according  to  the 
objectives  presented  previously.  The  final  chapter  presents  implica- 
tions of  the  results  and  some  ideas  for  future  research. 


CHAPTER  II 
THE  AUDIT  PROCESS  AND  PROFESSIONAL  JUDGMENT 

Introduction 

The  problem  of  interest  has  been  broadly  stated  as  a  lack  of 
understanding  of  the  impact  of  IAC  information  on  other  audit  proce- 
dures. The  sequential  nature  of  the  auditing  process  and  the  fact 
that  professional  judgment  plays  a  major  role  in  the  evaluation  of 
information  are  both  contributing  factors  to  this  problem.  The  pur- 
pose of  this  study  is  to  further  describe  the  impact  of  IAC  informa- 
tion on  the  auditor's  judgment  concerning  the  reliability  of  the  IAC 
system  and  his  resulting  choice  of  the  amount  of  substantive  testing. 

In  this  chapter  the  relevant  components  of  the  audit  process 
and  the  role  of  professional  judgment  are  introduced  and  discussed. 
Relevant  literature  in  these  areas  will  also  be  noted  and  in  some 
cases  discussed. 

The  Audit  Process 

Figure  2-1  presents  a  diagrammatic  representation  of  the  audit 
process.   Many  studies  in  auditing  have  dealt  either  with  one  of  these 
steps  exclusively  or  various  combinations  of  steps.   This  study  is 
concerned  with  the  decisions  required  at  the  two  key  points  represented 
by  Step  3  and  Step  5,  with  primary  interest  in  the  responses  of  the 
auditors  at  Step  5. 
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Figure  2-1.   The  audit  process. 
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Both  of  these  two  key  decision  points  within  the  audit  process 
call  for  professional  judgment  on  the  part  of  the  auditor.  First,  as  a 
result  of  internal  control  evaluation,  the  auditor  must  decide  on  the 
degree  of  reliability  he  associates  with  existing  controls  in  order  to 
develop  the  appropriate  preliminary  audit  program  (Step  3).  This  pro- 
gram would  include  both  compliance  and  substantive  tests.   Second,  and 

2 
more  importantly,   the  auditor  may  be  required  to  adjust  his  planned 

substantive  tests  (Step  5)  as  warranted  by  his  evaluation  of  the  results 
of  compliance  testing  (Step  4) . 

One  study  [Ashton,  1974]  has  particular  relevance  to  the  present 
study  since  it  deals  specifically  with  the  auditor's  ability  to  assess 
the  internal  control  environment  (Step  2)  and  the  extent  to  which 
auditors  agree  at  this  point  in  the  audit  process.   Other  studies 
[Joyce,  1976;  Mock  and  Turner,  1979J  have  expanded  the  decisions 
required  by  their  subjects  to  include  some  type  of  program  planning 
(Step  3),  such  as  a  specification  of  either  man  hours  or  sample  sizes.3 
The  present  study  will  extend  to  the  audit  program  revision  stage  (Step 
5)  of  the  audit  process. 

The  dynamic  nature  of  the  audit  environment  requires  the 
auditor  to  adjust  his  plans  in  response  to  additional  information.   The 


This  would  include  the  completion  of  the  internal  control 
questionnaire,  walk-through  of  the  system  and  its  evaluation. 

2 
The  second  decision  point  is  more  important  because  actual 
allocation  of  the  firm's  resources  results  from  this  decision. 

3 
These  three  studies  will  be  discussed  in  greater  detail  later 
in  this  chapter. 
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auditor  will  plan  the  nature,  timing,  and  extent  of  the  audit  procedures 
he  deems  most  appropriate  as  a  result  of  his  knowledge  about  the  client 
and  the  specific  objectives  of  the  audit.   The  auditor  is  then  required 
to  evaluate  additional  evidence  to  determine  if  a  revised  audit  program 
is  appropriate.   More  specifically,  additional  information  obtained 
through  the  performance  of  compliance  tests  on  internal  controls  and 
the  resulting  evaluation  of  such  tests  must  be  considered. 

The  importance  of  analyzing  the  program  revision  stage  of  the 
audit  process  was  recognized  by  Joyce  as  follows: 

The  selection  of  audit  program  planning  as  a  measure 
of  audit  work  to  perform  can  be  criticized  on  the 
grounds  that  auditors  may  disagree  on  an  audit  program 
yet  end  up  performing  the  same  tests  and  arriving  at 
the  same  opinion.   Differences  in  the  initial  audit 
programs  might  vanish  as  they  are  revised  in  view  of 
information  collected  as  the  audit  progresses.   I  am 
unaware  of  any  empirical  evidence  to  support  or  refute 
this  contention  [1976,  p.  35]. 

Although  individuals  and  accounting  firms  may  express  differ- 
ent audit  philosophies  and  may  prefer  a  different  emphasis  on  audit 
procedures,  the  essence  of  the  audit  process  as  reflected  in  Figure 
2-1  is  generally  accepted.   That  is,  evidence  is  gathered  in  a  sequen- 
tial process  that  allows  for  the  nature,  extent,  and  timing  of  such 
evidence  to  be  controlled  in  a  manner  that  is  deemed  optimal.   It  is 
the  evidence  gathering  and  evaluation  process  that  ultimately  results 
in  the  decision  as  to  the  optimal  allocation  of  resources  within  an 
audit.   Therefore,  this  decision  should  be  a  major  concern  of  those 
performing  the  audit.   The  importance  of  the  recognition  of  the 


4 
This  would  include  any  information  he  has  obtained  personnally 
in  either  the  current  or  previous  years  and  the  information  made  avail- 
able to  him  from  the  working  papers  relating  to  his  client. 
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sequential  nature  of  the  audit  process  has  been  highlighted  by  Wright 
[1976].   A  focus  of  this  study  will  be  on  the  sequential  nature  of 
the  audit  process  as  the  auditor  uses  his  professional  judgment  in 
making  the  decisions  relative  to  Step  3  (preliminary  audit  program 
planning)  through  Step  5  (audit  program  revision). 

The  importance  of  understanding  the  revision  stage  of  the 
audit  process  has  been  expressed  by  Mock  and  Turner  as  follows: 

Understanding  the  auditor's  decision  making  process 
may  lead  to  decision  aids  which  will  assist  the  auditor 
in  evaluating  audit  evidence.   It  certainly  seems 
unlikely  that  significant  improvements  will  be  forth- 
coming without  some  general  agreement  on  how  auditors 
act  when  faced  with  decisions  on  how  much  substantive 
evidence  is  appropriate  in  different  internal  control 
situations  [1979,  p.  277]. 

Internal  Control 

The  importance  of  internal  control  evaluation  and  its  impact 
on  the  ultimate  issuance  of  an  audit  opinion  has  led  to  many  recent 
research  efforts  relating  specifically  to  internal  control.   For 
example,  Burns  [1974],  Burns  and  Loebbecke  [1975],  and  many  others 
have  discussed  internal  control  evaluation  from  various  perspectives. 
Mathematical  approaches  to  the  analyzing  and/or  modeling  systems  of 
internal  control  have  been  presented  by  Grimlund  [1978],  Bodnar 
11975],  Cushing  [1974],  Yu  and  Neter  [1973],  and  others. 

A  certain  level  of  internal  control  is  expected  and  even 
required  to  be  present  within  a  firm  (see  The  Foreign  Corrupt  Practices 
Act  of  1977).   The  accumulation  of  additional  audit  evidence  is 
affected  directly  by  the  IAC  environment.   Of  interest  in  this  study 
is  the  relationship  between  internal  control  evaluation  and  subsequent 
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compliance  and  substantive  testing.   The  impact  of  IAC  evaluation  on 
audit  scope  adjustments  has  been  discussed  by  Morris  and  Anderson  [1976] 
and  Smith  [1972].   This  type  of  analysis  was  greatly  extended  by  Kinney 
[1975a;  1975b] .   Kinney  presented  a  decision  theory  approach  to  exami- 
nations of  the  relationship  of  IAC  evaluation  and  compliance  and 
substantive  tests  that  consider  such  relevant  factors  as  costs  of 
sampling,  costs  of  errors,  and  utility  functions  of  the  auditor.   How- 
ever, the  nature  of  the  decision  theory  approach  extends  beyond  the 
scope  of  the  present  study. 

Within  the  context  of  the.  present  study,  the  relationships 
between  the  key  factors  of  IAC  evaluation  and  substantive  testing  can 
be  related  to  the  concept  of  risk  and  summarized  as  follows: 

(1-R  )   =   (1-R  )  •  (1-R  ) 
o  C  S 

where,  (1-R  ):   Total  overall  risk  [R  =  overall  reliability] 
o  o 

(1-R  ):   Risk  of  substantive  tests  [R  =  reliability 
of  substantive  tests] 

(1-R  ) :   Risk  of  internal  control  [R  =  reliability 
of  internal  control] 

The  use  of  statistical  sampling  has  been  advocated  as  one  way  of 
controlling  for  overall  audit  risk  [Roberts,  1978;  Warren,  1979]. 

Statistical  Sampling  and  Professional  Judgment 

The  nature  of  the  evaluation  process  performed  by  auditors  and 
the  role  that  the  concept  of  risk  has  within  the  decision  process  has 
led  to  an  increasing  appreciation  by  auditors  of  statistical  sampling 
techniques. 
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Elliot  and  Rogers  [1972]  discussed  in  a  very  basic  fashion  the 
use  of  statistical  sampling  in  auditing.   Arkin  [1976]  analyzed  the  use 
of  statistics  in  auditing  with  particular  emphasis  upon  the  area  of 
internal  control  compliance.   Other  discussions  of  statistical  sampling 
in  auditing  include  Loebbecke  and  Neter  [1975]  and  Teitlebaum  and 
Robinson  [1975] . 

Regardless  of  the  extent  to  which  statistical  sampling  tech- 
niques are  employed,  the  importance  of  professional  judgment  is  not 
diminished.   The  use  of  statistical  sampling  for  planning  compliance 
and  substantive  tests  would  require  judgments  such  as  the  following: 
1)  expected  error  rates  in  the  population,  2)  desired  level  of  confi- 
dence, 3)  desired  precision  level,  4)  standard  deviation  of  population 
items,  and  5)  the  amount  considered  material.   The  role  of  judgment  in 
statistical  sampling  has  been  evaluated  by  Boer  [1974].   Uecker  and 
Kinney  [1977]  reported  results  of  a  study  to  investigate  the  type  and 
severity  of  errors  that  practicing  CPAs  make  in  the  judgmental  evalu- 
ation of  statistical  sampling  outcomes.   Results  indicated  that  the 
problem  of  subjectively  evaluating  sample  outcomes  may  be  significant. 
In  their  study,  seventy-four  percent  of  the  CPAs  made  at  least  one 
serious  error  of  judgment  and  fifty-six  percent  made  at  least  two  such 
errors. 

The  role  of  professional  judgment  becomes  paramount  as  statis- 
tical sampling  techniques  must  be  integrated  with  various  stages  of 
the  audit  process.   The  most  obvious  integration  occurs  within  internal 
control  assessment.   That  is,  the  auditor  must  decide  the  extent  to 
which  he  will  rely  upon  internal  controls  in  planning  the  nature, 
timing,  and  extent  of  compliance  and  substantive  tests.   Due  to  the 
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difficulty  in  dealing  with  all  of  these  planning  variables,  this  study 
will  emphasize  the  decisions  concerned  with  the  extent  of  testing. 

Given  the  role  of  professional  judgment  within  the  audit 
process  in  determining  the  extent  of  audit  work  to  perform,  the  com- 
mitment of  a  firm's  resources  could  vary  considerably  with  variations 
in  auditor  judgments.   In  the  absence  of  an  operational  normative 
model  to  rely  upon,  a  strict  evaluation  of  such  judgments  as  being 
"correct"  or  "optimal"  is  not  possible.   The  appropriate  evaluation 
criteria  then  becomes  the  degree  of  consensus  among  auditors  when 
presented  with  the  same  audit  environment  and  audit  evidence.   Einhorn 
[1974]  suggests  that  consensus  among  experts  is  one  of  several  neces- 
sary, although  not  sufficient,  conditions  for  the  existence  of  pro- 
fessional expertise. 

Although  the  actual  cost  associated  with  a  lack  of  consensus 
among  auditors  can  not  be  measured  directly,  various  forms  of  evidence 
are  available  that  suggest  the  auditing  profession  strives  for  consen- 
sus.  Such  factors  include  the  following:   1)  the  administration  of  a 
uniform  CPA  exam,  2)  issuance  of  standards  to  follow  in  the  practice 
of  auditing,  and  3)  the  use  of  training  schools  by  individual  firms  to 
guide  the  learning  process  of  their  employees.   Given  the  actions  taken 
to  strive  for  consensus  by  the  auditing  profession,  an  inference  can 
be  made  that  the  cost  of  such  actions  must  be  less  than  the  alternative 
cost  of  not  having  a  certain  degree  of  consensus. 

A  major  question  of  interest  then  becomes,  "where  within  the 
audit  process  should  this  consensus  be  reached?"   Should  consensus 
exist  at  every  step  of  the  audit  process,  or  is  agreement  as  to  the 
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opinion  that  is  to  be  issued  the  only  point  at  which  consensus  is 
important?  The  answer  to  this  question  seems  to  be  that  while  the 
issuance  of  the  opinion  is  a  key  point  of  consensus,  firms  are  also 
interested  in  the  optimal  utilization  of  their  available  resources  as 
they  seek  to  reduce  the  probability  of  making  an  incorrect  decision.5 
The  lack  of  consensus  at  decision  points  associated  with  the  commitment 
of  a  firm's  resources  could  result  in  a  less  than  optimal  use  of  such 
resources.   The  topic  of  consensus  and  other  topics  in  the  human 
information  processing  area  that  are  relevant  to  this  study  are  dis- 
cussed in  the  following  section. 

Human  Information  Processing 
Psychological  Literature 

The  area  of  human  information  processing  (HIP)  has  been 
examined  extensively  in  the  psychological  literature.   Three  broad 
approaches  to  the  study  of  HIP  include:   1)  Bayesian,  2)  regression, 
and  3)  cognitive  complexity.   A  number  of  authors  have  discussed  the 
relative  merits  of  the  different  approaches  to  study  HIP  [Slovic  and 
Lichtenstein,  1971;  Slovic,  Fischhoff,  and  Lichtenstein,  1977]  with 
comments  concerning  the  appropriate  use  of  each  of  these  approaches. 

A  Bayesian  approach  is  most  concerned  with  the  evaluation  of 
an  "optimal"  manner  to  combine  information,  or  "how  men  should  think." 

The  auditor  is  concerned  with  controlling  the  probability  of 
accepting  the  financial  statements  as  being  "fairly  presented"  when, 
in  fact,  they  are  not.   The  cost  of  this  Type  11(3)  error  could  be 
very  large. 
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Thus,  it  provides  a  normative  model  for  the  evaluation  of  probabilities 
expressed  by  judges  or  subjects.   A  cognitive  complexity  approach  to 
the  evaluation  of  HIP  requires  that  individuals  be  put  into  complex 
situations.   The  processing  of  information  is  then  evaluated  as  deci- 
sions are  made  to  determine  the  "style"  used. 

This  study  is  not  concerned  with  the  ability  of  the  subject  to 
"optimally"  process  information  (Bayesian  approach)  or  the  style  used 
(cognitive  complexity  approach).   Therefore,  the  regression  approach 
is  considered  most  appropriate  for  use  in  this  study,  with  the  "lens 
model"  considered  the  most  applicable  "representative  design"  within 
the  regression  approach. 

The  lens  model   is  based  on  the  assumption  that  man  makes 
inferences  about  the  state  of  an  uncertain  environment  on  the  basis  of 
uncertain  information.   Therefore,  the  outcomes  that  result  from  an 
interaction  between  man  and  his  uncertain  environment  are  most  cer- 
tainly probabilistic.   The  lens  model  specifies  a  relationship  between 
the  probabilistic  outcomes  of  the  environment  and  the  prediction  or 
judgment  by  an  individual  of  a  particular  state  of  that  environment. 
Analysis  of  the  information  represented  on  the  "right-hand- 
side"  of  the  lens  model  (the  actual  and  predicted  subjects'  responses) 
can  result  in  the  capturing  of  the  decision  policy  of  the  subjects. 
The  correlation  of  the  actual  responses  between  the  subjects  is  con- 
sidered an  appropriate  measure  of  consensus  (agreement).   Also,  the 
mathematical  modeling  of  the  judgment  process  allows  for  the  evaluation 


For  a  more  complete  presentation  and  explanation  of  the  lens 
model  refer  to  Chapter  III,  Brunswik  {1949],  or  Slovic  and  Lichtenstein 
[1971]. 
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of  the  relative  importance  of  the  various  cues  (factors).   Self- 
insight  indices  can  then  be  obtained  by  correlating  the  mathematical 

weightings  of  the  cues  with  the  subjective  weightings  provided  by 
the  subjects. 

The  following  sections  present  a  discussion  of  the  evaluation 

criteria  of  consensus  and  self-insight  that  were  mentioned  above. 

These  particular  areas  have  been  investigated  in  the  psychological 

literature  and  have  the  most  relevance  to  this  study. 

Consensus.   Consensus  is  the  extent  of  agreement  among  judges 
using  the  same  information  at  the  same  point  in  time.   Many  psycho- 
logical studies  have  indicated  that  consensus  among  judges  is  very 
poor  (e.g.,  Goldberg,  1968):  most  accounting  studies  have  indicated 
a  relatively  high  degree  of  consensus. 

The  extent  of  consensus  found  in  most  studies  is  thought  to 
be  a  function  of  the  judgment  task  and  the  specific  environment.   How- 
ever, most  realistic  tasks  have  yielded  results  which  indicate  a  lack 
of  consensus  among  persons  considered  professionals  with  respect  to 
the  judgmental  task.   Various  opinions  exist  as  to  the  source  of  such 
differences.   For  example,  Ashton  and  Kramer  state  the  following: 

Some  psychologists  (e.g.,  Brehmer,  1976;  Dawes,  1975; 
Edwards,  1971)  have  argued  recently  that  certain  formal 
or  statistical  characteristics  of  the  judgment  task, 
and  not  the  substantive  or  content  characteristics  of 
the  task  nor  the  particular  subjects  in  the  task,  are 
the  most  important  determinants  of  human  information 
processing  and  decision  making  [1980]. 

Self -insight.   Self-insight  is  the  ability  of  the  judge  to 
determine  the  relative  weights  of  the  various  factors  he  uses  in  his 
judgment  process.   Studies  have  indicated  that  judges  exhibit  poor 
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insight  into  their  own  weightings  of  factors  [Slovic,  1969;  Slovic, 
Fleissman,  and  Bauman,  1972].   Researchers  have  found  differences 
between  the  subjective  weights  provided  by  judges  and  those  computed 
by  a  mathematical  model  of  their  judgmental  policies.   For  example, 
the  results  from  psychological  studies  have  indicated  that  judges  over- 
estimate the  less  important  factors  and  underestimate  the  more  important 
factors  (e.g.,  Slovic,  1969). 

Of  additional  interest  is  the  finding  that  non-linear  models 
of  the  judgment  process  contribute  little  as  compared  to  linear  models 
[Goldberg,  1968;  Slovic,  1969;  Slovic  and  Lichtenstein,  1971;  Dawes 
and  Corrigan,  1974J .   The  implication  is  that  a  simple  linear  model 
will  satisfactorily  approximate  the  manner  in  which  an  individual 
processes  information.   The  adequacy  of  the  linear  model  also  will  be 
examined  in  the  current  study. 

Auditing  Literature 

Auditing  researchers  also  have  developed  an  interest  in  HIP, 
the  applicability  of  the  lens  model  approach,  and  the  resulting  model 
of  the  judgment  process.   The  probabilistic  relationship  between  man 
and  his  uncertain  environment,  as  reflected  in  the  lens  model,  is 
certainly  present  in  the  audit  environment. 

The  auditor  is  required  to  evaluate  the  uncertain  audit 
environment  to  ascertain  that  either  "acceptable"  or  "unacceptable" 
situations  exist.   He  must  then  make  decisions  concerning  specific 
combinations  and  the  extent  of  audit  procedures  that  are  most  appro- 
priate given  his  current  beliefs  about  the  state  of  the  environment. 
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Within  the  specific  judgmental  context  of  this  study,  the  true 
state  of  the  environment  is  that  either  a  specific  control  objective  is 
"acceptably"  achieved,  or  it  is  not.   Of  interest  in  this  study  is  the 
extent  of  evidence  required  by  the  auditors  to  determine  whether  an 
objective  has  been  "acceptably"  achieved.   Because  the  "true"  state  of 
the  environment  cannot  be  determined,7  the  analysis  of  this  study  is 
limited  to  the  "right-hand-side"  of  the  lens  model  where  the  auditor's 
judgment  and  his  use  of  cues  can  be  evaluated.   A  better  understanding 
of  this  approach  is  obtained  through  a  discussion  of  some  recent  studies 
in  audit  ing . 

Most  Relevant  Prior  Research 

Various  studies  have  evaluated  the  extent  of  consensus  among 
auditors  at  different  points  in  the  audit  process.   Results  indicate 
that  the  degree  of  consensus  among  auditors  apparently  varies  according 
to  the  particular  stage  in  the  audit  process  or  according  to  the  type 
of  question  addressed.   Ashton  [1974]  and  others  who  have  replicated 
and  extended  his  original  study,  Joyce  [1976],  and  Mock  and  Turner 
11979],  provide  a  basis  for  summarizing  the  judgment  literature  in 
accounting.   These  studies  will  be  discussed  separately,  with  particu- 
lar emphasis  on  their  relationship  to  the  present  study. 

Ashton  Study,  Replications  and  Extensions 

Ashton  was  most  concerned  with  the  extent  of  inconsistency 
shown  by  auditors  in  their  evaluation  of  internal  control.   By 

For  relevant  discussions  of  the  importance  of  the  "true"  or 
[19761i0n  V3lUe  ln  the  lenS  m°del  anSlysis'  See  Casey  [1976]  and  Ashton 
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presenting  sixty-three  practicing  auditors  with  thirty-two  cases  (each 
a  different  combination  of  Yes/No  responses  to  six  questions  found  on 
an  internal  control  questionnaire  for  payroll),  judgments  of  internal 
control  strength  were  elicited.   These  judgments  were  made  on  a  scale 
from  one  (extremely  weak)  to  six  (adequate  to  strong) . 

Ashton  used  a  descriptive  analysis-of-variance  (ANOVA)  tech- 
nique to  evaluate  two  types  of  consistency.   These  are:   1)  consensus, 
which  refers  to  consistency  across  auditors  at  the  same  point  in  time, 
and  2)  stability,  which  refers  to  consistency  over  time  for  the  same 
data.   By  correlating  the  judgments  of  each  auditor  on  the  cases,  both 
with  the  other  auditors  and  with  later  administrations  for  the  same 
auditor,  Ashton  found  that  the  judgments  exhibited  a  "fairly  high  level' 
of  consensus  (r=.70),  stability  (r=.81),  and  self-insight  (r=.89). 
By  addressing  the  very  crucial  question  of  "to  what  extent  do 
auditors  agree  in  their  assessment  of  the  strength  of  an  internal 
control  questionnaire?"  Ashton 's  study  represents  a  major  contribution 
to  auditing  research.   Ashton  chose  to  replicate  his  study  with  the 
use  of  upper  division  auditing  students  at  The  University  of  Texas  at 
Austin  [Ashton  and  Kramer,  1980J .   These  results  were  inconclusive 
with  respect  to  whether  the  auditing  students  were  good  surrogates 
for  practicing  auditors.   However,  the  results  did  indicate  a  similar 
high  degree  of  consensus  on  the  part  of  the  students  (r=.66),  with 
auditors  also  showing  a  higher  degree  of  self- insight  (r=.89)  as 
compared  to  the  students  (r=.77). 
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Ashton  and  Brown  [1980]  extended  Ashton's  earlier  work  through 
the  addition  of  two  cues  to  the  six  that  had  been  used  in  the  original 
study.   The  subjects  used  in  this  study  were  31  practicing  auditors 
from  four  offices  of  seven  of  the  "Big-Eight"  public  accounting  firms. 
A  similar  consensus  index  was  found  (r=.67),  with  self-insight  indices 
averaging  .86.   It  should  be  noted  that  this  lower  average  self-insight 
index  was  obtained  from  auditors  who  were  generally  less  experienced 
than  those  who  participated  in  the  original  study,  but  who  were  of 
course  more  experienced  than  the  students  tested  in  the  Ashton  and 
Kramer  study. 

Hamilton  and  Wright  [1977]  also  used  the  payroll  environment 
to  evaluate  audit  judgment  in  a  modified  replication  of  Ashton's 
original  study.   Seventeen  practicing  auditors  from  the  same  city  were 
presented  with  five  internal  control  questions  relative  to  payroll. 
The  average  values  of  consensus  and  self -insight  were  .66  and  .87, 
respectively.   Interesting  results  with  respect  to  experience  levels 
showed  that  the  group  with  more  experience  had  both  greater  insight 
(.93  to  .84)  and  greater  consensus  (.78  to  .62).   These  particular 
findings  are  consistent  with  the  results  of  the  Ashton  and  Ashton  and 
Kramer  studies. 

The  most  recent  extension  of  Ashton's  study  of  internal  control 
relative  to  payroll  was  provided  by  Reckers  and  Taylor  [1979].   Thirty 
practicing  auditors  were  presented  with  a  completed  payroll  question- 
naire containing  responses  to  approximately  36  questions:   these 
responses  were  varied  to  form  5  different  cases.   In  analyzing  the 
subjects'  responses  as  to  the  reliability  of  the  internal  control 
system  (0-100  percent),  an  average  inter-rater  correlation  of  .155 
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was  obtained.   This  increased  slightly  (to  .357)  for  those  participants 
with  more  than  seven-and-one-half  years'  experience.   These  results 
suggest  that  the  level  of  consensus  is  reduced  as  greater  realism  is 
introduced  into  the  audit  task.   When  auditing  professors  were  pre- 
sented with  the  same  task  similar  results  were  obtained  (r =  .28). 
However,  Ashton  [1979b]  expresses  sone  concern  about  the  cases  used 
and  assumptions  made  by  Reckers  and  Taylor. 

The  studies  discussed  above  provided  a  meaningful  evaluation 
of  audit  judgment  within  the  payroll  environment  and  with  respect  to 
the  strength  of  an  internal  control  environment.   However,  because  of 
their  nature,  extensions  of  these  studies  appear  warranted.   A  natural 
progression  of  auditor  judgment  research  would  involve  a  shift  in 
focus  to  a  different  point  in  the  audit  process.   This  shift  appears 
necessary   since  the  conclusion  that  auditors  have  exhibited  a  "fairly 
high  level"  of  consensus  relative  to  the  strength  of  internal  control 
says  nothing  as  to  the  extent  of  their  agreement  at  a  later  point  in 
the  audit  process;  or  more  specifically,  as  to  how  this  level  of 
agreement  extends  to  decisions  about  the  extent  of  other  audit  proce- 
dures to  perform.   Joyce  provided  such  an  extension. 

Joyce  Study 

Joyce  [1976J  extended  the  research  in  the  area  of  auditor 
judgment  by  focusing  upon  the  decision  regarding  the  extent  of  audit 
work  (planned  man-hours)  rather  than  upon  the  assessments  of  the 
quality  of  internal  control.   This  represented  an  analysis  of  auditor 
judgment  for  a  different  question  at  a  later  point  in  the  audit 
process. 
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Thirty-five  practicing  auditors  were  presented  with  a  set  of 
sixteen  systematically  varied  combinations  of  stimulus  information 

Q 

related  to  an  audit  program  for  accounts  receivable.    The  auditors 
were  asked  to  indicate  the  planned  extent  (in  man-hours)  of  five  audit 
procedures. 

Some  conclusions  resulting  from  Joyce's  use  of  correlation 
measures  and  ANOVA  (or  MANOVA)  can  be  summarized  as  follows: 

(1)  There  was  considerable  disagreement  concerning  how  the 
independent  variables  (factors  used  in  determining  the 
extent  of  audit  procedures)  should  be  weighted. 

(2)  There  was  little  consensus  (r  =  .37)  among  auditors  con- 
cerning how  much  time  should  be  planned  for  the  audit 
procedures. " 

(3)  Main  effects  accounted  for  virtually  all  the  reliable 
judgment  variance. 

(4)  The  level  of  consensus  decreased  as  experience 
increased. 

The  stimulus  information  provided  by  Joyce  was  presented  as 
having  been  developed  from  various  sources  such  as  an  internal  control 
questionnaire  and  analytical  review  data  (i.e.,  significant  ratios  and 
turnovers).   An  appropriate  adjustment  to  this  experimental  environment 
would  be  to  provide  the  auditors  with  a  more  realistic  setting  and 
information  as  to  the  results  of  compliance  tests  on  the  internal 


Joyce  actually  performed  two  experiments,  with  the  second 
experiment  being  a  full  replication  of  the  25  factorial  conducted  as 
Experiment  1.   Joyce  was  therefore  able  to  assess  the  impact  of  two- 
way  and  three-way  interactions. 

9 
Note  that  although  Joyce's  subjects  made  multiple  judgments 
on  each  case,  the  consensus  measure  was  calculated  by  summing  the 
audit  hours  on  each  case  and  then  correlating  the  subjects'  judgment. 
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control  information.   Mock  and  Turner  performed  a  study  with  these 
characteristics. 

Mock  and  Turner  Study 

Mock  and  Turner  [1979J  sought  empirical  evidence  about  the 
effect  of  changes  in  internal  controls  on  auditors'  decisions  on  the 
extent  of  substantive  testing.   Seventy-one  audit  seniors  and  two  audit 
supervisors  (all  from  a  single  firm)  were  given  information  on  improve- 
ments in  internal  controls  from  the  previous  year  and  asked  to  adjust 
the  sample  sizes  for  four  specific  audit  procedures  from  the  planned 
audit  program.   The  improved  internal  controls  were  evidenced  primarily 
by  a  general  change  in  compliance  test  results  of  specific  controls  for 
the  current  year  as  compared  to  the  previous  year. 

Some  of  the  major  results  of  the  study  include  the  following: 

(1)  Subjects  reduced  their  judgmental  samples  in  every  case 
except  one.   (The  subjects  were  dealing  only  with  improve- 
ments in  controls  and  the  exception  was  probably  due  to 
the  fact  that  the  planned  sample  for  this  case  was 
knowingly  set  quite  low.) 

(2)  Subjects  reacting  to  strong  controls  consistently  recom- 
mended smaller  sample  sizes  for  planned  substantive  tests 
than  did  subjects  reacting  to  fair  controls. 

(3)  There  was  considerable  variability  in  auditors'  decisions. 
(Results  showed  both  wide  ranges  and  large  coefficients 

of  variation  for  all  control  procedures  with  sample  size 
decisions  for  strong  controls  varying  much  less  than  for 
fair  controls.) 

(4)  No  evidence  was  found  of  a  relationship  between  the  sub- 
jects' backgrounds  and  their  judgments. 
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(5)   Possible  effects  of  anchoring  °  were  found.   Those  who 
had  anchors  recommended  smaller  sample  sizes  and  men- 
tioned in  rationale  memos  that  they  gave  attention  to 
the  previous  sample  sizes. 

The  present  study  is  intended  to  be  an  important  extension  of 
the  study  by  Mock  and  Turner.   The  decision  to  build  upon  and  to  extend 
their  study  is,  to  some  extent,  based  upon  their  use  of  a  realistic 
setting.   Accordingly,  some  specific  characteristics  of  Mock  and  Tur- 
ner's experimental  environment  and  important  extensions  and  differences 
relative  to  this  study  are  provided  below.   These  particular  character- 
istics of  Mock  and  Turner's  study  may  be  summarized  as  follows: 

(1)  They  provided  general  evidence  about  changing  internal 
control  strengths  (resulting  from  compliance  tests)  on 
a  year-to-year  basis. 

The  present  study  provides  more  specific  information  pertaining 
to  the  results  of  compliance  testing  of  key  internal  controls  as  com- 
pared to  the  preliminary  evaluation  of  the  current  period.   These 
specific  results  will  then  serve  as  a  basis  for  adjusting  the  planned 
extent  of  audit  procedures  for  the  current  period.   This  restriction 
will  serve  to  reduce  the  possibility  of  incorporating  confounding 
variables  into  the  experimental  setting,  through  superior  control  over 
the  factors  being  considered  by  the  auditor. 

(2)  They  required  the  subjects  to  deal  with  a  "perceived" 
audit  objective  in  a  broad  audit  setting  (i.e.,  the 
Revenue  Cycle) . 


"Anchoring"  is  the  process  of  adjusting  from  initial  values 
or  starting  points  to  yield  final  estimates.   See  Tversky  and  Kahneman 
[1974]  for  a  discussion  of  this  heuristic  principle. 
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This  study  also  uses  the  Revenue  (Sales)  Cycle  as  the  experi- 
mental environment.   However,  a  specific  audit  objective  is  provided 
to  the  subjects  and  the  internal  control  information  is  then  keyed  to 
this  particular  objective.   This  will  allow  the  auditors  to  evaluate 
the  evidence  provided  to  them  on  a  "common"  basis  and  therefore  make 
the  comparison  of  their  decisions  more  valid. 

(3)  The  subjects  were  told  the  degree  of  reliance  to  place 
on  internal  controls  for  either  fair  or  strong  controls. 

The  subjects  in  this  study  are  provided  with  the  specific 
results  of  compliance  tests  (noncompliance  rates  for  the  sample  sizes 
selected  by  the  subjects)  and  must  decide  their  degree  of  reliance  on 
the  key  internal  controls  after  receiving  these  results.   This  adjust- 
ment incorporates  additional  realism  into  the  setting  and  provides 
greater  assurance  as  to  the  message  that  is  being  provided  to  the 
auditor.   The  auditor  can  then  interpret  this  message  as  he  desires. 

(4)  The  subjects  were  presented  only  with  improvements  in 
internal  controls  and  asked  to  adjust  a  planned  sample 
size. 

This  study  provides  various  combinations  of  "confirming"  and  "discon- 
firming"  evidence  as  to  the  existence  and  effectiveness  of  the  key 
internal  controls  as  compared  to  certain  expectations.   The  subjects 
also  are  asked  to  adjust  their  own  planned  sample  size  for  a  substan- 
tive test  rather  than  to  adjust  a  planned  sample  size  that  has  been 
provided  to  them.   As  this  study  is  most  concerned  with  the  final 
sample  size  decision,  it  was  felt  that  providing  a  planned  sample  size 
would  influence  the  auditors'  decisions.   Again,  a  confounding  factor 
would  be  introduced  because  it  would  not  be  clear  what  effect  this 
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would  have  on  each  auditor.   It  was  considered  that  an  inferior 
measure  of  consensus  among  auditors  would  be  obtained  if  a  planned 
sample  size  was  provided. 

(5)  The  subjects  were  presented  with  only  one  combination 
of  controls  to  use  in  planning  a  substantive  program. 

The  subjects  in  this  study  are  provided  with  twelve  combinations 
of  compliance  test  results  for  three  key  internal  controls  that  are 
considered  most  important  in  meeting  the  specific  objective.   This  is 
necessary  to  allow  for  the  desired  analysis  relative  to  consensus  and 
the  weightings  of  the  key  variables.   The  single  case  approach  taken 
by  Mock  and  Turner  did  not  allow  for  a  within-subject  evaluation. 

(6)  The  subjects  were  required  to  make  "judgmental"  sample 
size  decisions. 

All  sample  sizes  in  this  study  are  chosen  from  statistical 
sampling  tables  provided  to  the  subjects.   For  compliance  tests, 
tables  were  provided  that  gave  the  subjects  various  confidence  levels 
and  desired  precision  levels  from  which  to  choose.   For  the  substan- 
tive test,  a  table  that  was  derived  from  a  difference  estimation 
approach  was  provided;  in  using  this  table  it  is  necessary  for  the 
subject  to  decide  upon  a  confidence  level  and  an  amount  for  material- 
ity.  This  approach  provides  additional  information  as  to  which 
factors  were  considered  most  important  and/or  altered  as  the  auditor 
moved  from  case  to  case. 

The  above  discussion  provides  a  comparison  of  this  study  with 
the  most  recent  study  of  Mock  and  Turner.   Perhaps  the  most  obvious 
intention  of  this  study  is  to  provide  the  auditor  with  a  more  compact 
audit  environment  and  specific  audit  objective  to  a  greater  degree 
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than  in  previous  studies.   At  the  same  time,  the  desire  to  present  a 
realistic  setting  is  also  of  concern. 

Summary 

The  review  of  the  psychology  and  auditing  literature  presented 
in  this  chapter  provides  additional  support  for  the  contention  that 
the  application  of  professional  judgment,  both  in  general  and  within 
the  auditing  context,  deserves  further  investigation.   Support  is  also 
provided  for  the  importance  of  the  evaluation  of  internal  accounting 
controls  and  their  role  within  the  audit  process. 

Various  deficiencies  in  previous  auditor  judgment  studies  were 
presented  and  discussed.   General  characteristics  of  this  study  were 
presented  as  significant  improvements  over  prior  studies.   Specific 
details  of  the  experiment  that  was  conducted  are  presented  in  Chapter 
III  in  conjunction  with  a  presentation  of  the  methodology  and  experi- 
mental design. 


CHAPTER  III 
THE  EXPERIMENT 

Introduction 

The  purpose  of  this  chapter  is  to  present  the  methodology  and 
experimental  design  used  in  this  study,  along  with  the  details  of  the 
experiment.   First,  in  order  to  provide  a  better  understanding  of  the 
descriptive  approach  used  in  this  study,  a  brief  description  of  the 
Brunswik  lens  model  and  the  analysis  of  variance  model  is  presented. 
A  discussion  of  the  specific  task,  the  experimental  design,  and  the 
methodology  follow  in  the  next  section.   Finally,  the  steps  in  the 
administration  of  this  study  and  some  expectations  as  to  results  are 
presented,  followed  by  several  recognized  limitations  of  the  study. 

The  Lens  Model 

The  lens  model  as  developed  and  described  by  Egon  Brunswik 
[1943,  1949]  has  become  an  acceptable  and  popular  model  to  use  in 
human  information  processing  (HIP)  research.   Because  of  the  descrip- 
tive nature  of  this  study,  the  lens  model  provides  the  most  appro- 
priate framework  for  the  description  and  examination  of  the  auditor's 
judgment  process. 

The  lens  model  is  considered  a  significant  development  within 
a  "correlational"  paradigm  of  the  regression  approach  to  the  study  of 


34 


35 


human  information  processing  [Slovic  and  Lichtenstein,  1971].   Bruns- 
wik's  model  emphasizes  "the  probabilistic  interrelations  between 
organismic  and  environmental  components  of  the  judgment  situation" 
[Slovic  and  Lichtenstein,  1971,  p.  655]  as  opposed  to  a  focus  on  the 
judge  (information  processor). 

Specification  of  the  Univariate  Lens  Model 

As  shown  in  Figure  3-1,  the  univariate  lens  model  utilizes 
both  environmental  and  subject  response  data.   The  left  side  of  the 
model  represents  the  environmental  system,  while  a  subject's  (decision 
maker's)  actual  and  predicted  responses  are  represented  on  the  right 
side.   Correlational  measures  are  indicated  for  the  relationships 
within  and  between  those  systems.   The  variables  shown  in  the  lens 
model  are  defined  as  follows: 

X_^  =   the  variables  found  in  the  information  set  which 
serve  as  cues  to  the  subject. 

Y^  =   the  criterion  value  or  "distal  variable."  This 

value  represents  the  actual  result  of  an  environ- 
mental event. 

Yg  =  the  predicted  criterion  value.   This  prediction 

results  from  taking  the  "optimal"  linear  combination 

of  the  cues,  as  accomplished  by  minimizing  the  squared 

deviations  between  Y  and  Y  . 
e      e 

k 

Therefore,  Y   =   I  b  X  , 
e     .  .   ie  i 
1=1 

where  b   represents  the  relative  weightings  of  the  cues. 

Ys   =  The  subject's  response.   This  value  represents  his 
judgment  as  to  the  criterion  value. 

Y^  =   the  predicted  response  of  the  subject.   This  prediction 
results  from  taking  the  "optimal"  linear  combination  of 
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the  cues,  as  accomplished  by  minimizing  the  squared  devia- 
tions between  Y  and  Y  . 
s      s 

k 

Therefore,  Y   =   Z  b  X  , 
s     ,  .   is  1 
i=l 

where  bis  represents  the  relative  importance  of  the 
cues  to  the  subject. 

The  "correlational"  nature  of  the  lens  model  is  reflected  by 

the  six  correlations  that  can  be  calculated  among  both  the  actual  and 

predicted  environmental  and  subject  response  variables.   Within  the 

environmental  system  (left  side)  of  the  model,  two  correlational 

measures  can  be  calculated  between  the  criterion  value  (Y  )  and  the 

e 

cues  (X.),  and  between  the  criterion  value  (Y  )  and  the  predicted 
i  e         r 

criterion  value  (Y  ) .   The  correlations  are  explained  as  follows: 

r     =   the  correlation  across  stimuli  between  cue  Xi  and 

Ye.   This  correlation  reflects  the  relevance  of  the 
ith  information  source  (cue)  in  the  environment. 

YeYe  =  the  correlation  between  Ye  and  Ye.   This  correlation 
represents  the  degree  to  which  the  (linearly) 
weighted  combination  of  cues  serves  to  predict  the 
state  of  Ye  and  is  referred  to  as  "environmental 
predictability. " 

Two  correlational  measures  can  also  be  calculated  within  the 

subject's  system  (right  side  of  the  lens  model).   The  subject's 

response  (Y  )  can  be  correlated  with  the  cues  (X  )  and  also  with  the 
s  l 

predicted  subject  response  (Y  ).   Explanations  of  these  relationships 
are  as  follows: 

rig  =  the  correlation  across  stimuli  between  cue  X^  and 
Ys.  This  correlation  represents  the  relevance  of 
the  ith  information  source  (cue  to  the  subject). 

TY  Y  =   the  correlation  between  Ys  and  Yg.   This  correlation 
s  s    represents  the  extent  to  which  the  subject's  judgments 
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can  be  predicted  by  a  linear  combination  of  cue 
values  and  is  referred  to  as  "response  linearity." 

An  evaluation  of  the  subject's  performance  can  be  obtained 
through  correlations  between  the  left  side  and  right  side  of  the  model. 
These  two  measures  are  known  as  the  "achievement  index"  and  the  "match- 
ing index"  and  are  explained  as  follows: 

rY  Y  =   the  correlation  between  Ye  and  Yg.   This  correlation 
e  s    represents  the  subject's  ability  to  predict  an  out- 
come and  is  known  as  an  "achievement  index." 

r^    ~  ys  ^ 

YeYs  =   the  correlation  between  Ye  and  Yg.   This  correlation 
is  between  the  two  regression  equation  model  esti- 
mates and  is  known  as  a  "matching  index." 

The  use  of  the  lens  model  in  the  current  internal  control 
evaluation  study  requires  the  presentation  of  a  set  of  relevant 
accounting  information  cues  (X.)  as  a  basis  for  the  auditor  (subject) 
to  exercise  his  judgment  in  deciding  the  extent  of  substantive  test- 
ing to  perform  (Yg) .   The  actual  decision  that  should  be  made  using 
the  information  cues  and  the  appropriate  loss  function  is  represented 
by  Yg.   Because  an  implied  loss  function  is  necessary  to  decide  an 
"optimal"  value  for  Y&,    this  value  is  not  known  within  the  context  of 
the  current  study. 

The  lack  of  a  true  criterion  value  (Y  )  is  not  unusual  for 

e 

real-world  judgment  situations.   As  a  result,  the  current  study 

concentrates  on  the  right  side  of  the  lens  model  as  the  framework  for 

analysis.   However,  the  emphasis  on  the  auditor's  responses  (Y  )  and 

s 

his  predicted  responses  (Y  )  does  provide  the  necessary  relationships 
to  properly  pursue  the  objectives  of  the  study.   It  could  be  argued  that 
by  obtaining  a  certain  level  of  consensus  among  the  auditors  in  this 
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study,  an  appropriate  value  for  Y  could  be  inferred  for  the  specific 
internal  control  environment  that  was  predicted. 

The  Multivariate  Lens  Model 

The  existence  of  more  than  one  criterion  value  within  the 
context  of  a  single  judgmental  situation  would  require  some  adjust- 
ment to  the  univariate  model  just  presented.   Within  the  current  study, 
the  auditor  must  implicitly  decide  upon  the  reliability  of  the  internal 
control  system  prior  to  selecting  an  appropriate  sample  size  for  the 
substantive  test.   Figure  3-2  presents  the  lens  model  representation 
of  a  two  criterion  value  situation. 

The  multivariate  lens  model  allows  for  the  simultaneous  evalu- 
ation of  a  judgment  process  when  there  is  more  than  one  criterion 
value.   An  obvious  difficulty  associated  with  the  use  of  the  univariate 
model  in  such  a  situation  has  been  summarized  as  follows:   "The 
researcher  would  be  left  with  a  large  number  of  indices  and  measures 
which  would  require  some  sort  of  aggregation  in  order  to  interpret 
exactly  what  was  occurring  in  the  entire  system"  [Castellan,  1972, 
p.  244]. 

Representation  of  the  two  criterion  value  case  requires  only 
minor  changes  to  vector  notation  as  follows: 

Y^  =  the  vector  (Y    Y  ). 

Y   =   the  vector  (Y  , ,Y  „) . 
— s  si   s2 

X   =   the  vector  (X  ,X  ,  .  .  .  ,X  ) 


The  following  is  taken  from  a  more  complete  discussion  of  a 
multiple  criterion  case  of  the  lens  model  discussed  by  Castellan 
[1972]. 
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The  multivariate  lens  model  for  the  two  criterion  case  is  then  presented 
as 

X*   =   (Y  ,Y  ,X), 
—       _e  _s  _ 

where  X*  is  a  representation  of  the  complete  system. 

The  appropriate  correlational  measures  then  become  multivariate 
and  can  be  obtained  from  the  use  of  canonical  correlation.   However, 
the  multivariate  analysis  does  not  affect  the  situation  where  a  true 
criterion  value  is  lacking.   Therefore,  in  the  current  study,  appro- 
priate canonical  correlations  are  found  relative  to  the  right  side  of 
the  lens  model. 

Analysis  of  Variance 

Strict  adherence  to  the  lens  model  framework  discussed  above 
would  not  allow  for  the  incorporation  of  a  nonlinear  model.   The 
analysis  of  variance  (ANOVA)  model  enables  the  examination  of  both 
linear  and  configural  processing  by  subjects.   The  extent  to  which  a 
cue  contributes  to  a  subject's  response  can  be  determined  through  an 
ANOVA  model,  as  can  the  significance  of  the  contribution  of  two  or 
more  cues  when  considered  simultaneously.   Also,  a  multivariate 
analysis  of  variance  (MANOVA)  is  possible  for  situations  having  more 
than  one  dependent  variable. 

The  ANOVA  (or  MANOVA)  framework  offers  some  other  distinct 
advantages  over  a  strict  adherence  to  the  lens  model  approach.   The 
use  of  the  factorial  design  and  the  orthogonality  obtained  are  specific 
advantages.   The  fact  that  the  cues  (X.)  can  be  discrete  or  categori- 
cal rather  than  continuous  offers  another  advantage  for  most  studies. 
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And  finally,  given  the  desire  to  examine  the  auditor's  ability  to 
understand  his  decision  process,  the  calculation  of  a  self-insight 
index  is  most  important.   This  is  possible  through  the  calculation  of 
omega-squared  indices  which  provide  an  indication  of  the  proportion 
of  total  variance  accounted  for  by  a  particular  cue  or  combination  of 
cues.   Therefore,  the  framework  of  the  current  study  is  expanded  to 
incorporate  the  ANOVA  model  to  provide  additional  important  analyses. 

The  Experimental  Design  and  Methodology 
The  Task 

Task  selection.    The  judgment  process  used  by  auditors  in 
routine  audit  situations  is  best  examined  through  the  use  of  a  specific 
audit  task.   The  experimental  task  chosen  for  this  study  requires  the 
subject  to  determine  the  reliability  to  be  attached  to  a  set  of  inter- 
nal controls  of  a  hypothetical  company,  and  then  to  use  the  available 
information  about  these  controls  in  determining  the  extent  of  substan- 
tive testing  to  be  performed  to  meet  a  specific  audit  objective  (e.g., 
to  satisfy  himself  that  recorded  sales  are  for  valid  transactions). 

As  discussed  in  Chapter  II,  previous  studies  dealing  with 
auditor  judgment  have  dealt  with  a  broader  audit  environment  and  there- 
fore a  somewhat  different  audit  task.   The  experimental  task  used  in 
this  study  was  chosen  because: 

(1)   The  task  is  most  representative  of  the  type  of  judgments 
that  auditors  must  make  at  different  points  within  the 
entire  audit  process. 
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(2)  It  incorporates  an  area  (internal  control  evaluation) 
that  has  become  a  most  important  topic  within  the 
auditing  profession. 2 

(3)  It  represents  a  very  important  portion  of  the  audit 
process  because  of  the  commitment  of  the  firm's  resources 
and  the  risks  associated  with  incorrect  decisions. 

(4)  The  task  can  be  represented  to  subjects  in  a  setting  that 
allows  for  the  best  combination  of  a  realistic  situation 
with  the  necessary  control  for  variables  not  of  specific 
interest  to  the  experimenter. 

(5)  It  deals  with  a  specific  objective  within  a  particular 
cycle  of  the  business.   (A  cycle  approach  to  auditing  is 
considered  an  acceptable  and  desirable  method  within  the 
profession. ) 

Task  development.    This  study  focuses  on  auditors'  decisions 
concerning  the  reliability  of  an  IAC  system  and  sample  sizes  for  com- 
pliance and  substantive  tests.   These  decisions  are  made  with  respect 
to  a  specific  audit  objective  (e.g.,  satisfaction  that  recorded  sales 
are  for  valid  transactions)  and  three  key  IACs  that  can  be  summarized 
as  follows:   (1)  proper  approval  for  credit,  (2)  authorization  for 
shipment,  and  (3)  control  over  physical  shipment  of  goods.   Since 
each  of  these  internal  controls  could  or  could  not  exist,  there  are 
eight  (2x2x2)  possible  combinations  that  could  be  presented.   However, 
for  practical  reasons,  a  number  of  these  "scenarios"  were  not  included 
in  the  experiment. 

First,  since  the  study  is  concerned  with  the  effect  of  compli- 
ance test  results,  the  situation  where  none  of  the  three  controls 
exist  was  not  considered  appropriate.   This  situation  also  appears  to 
be  "unrealistic"  in  terms  of  the  absence  of  all  three  of  the  key  controls. 

2 
See,  for  example,  the  AICPA  report  by  the  Commission  on  Audi- 
tor's Responsibility  [1978]  and  The  Foreign  Corrupt  Practices  Act  of 
1977. 
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The  objective  of  providing  settings  that  would  be  acceptable  as  realistic 
was  a  major  consideration  in  the  selection  of  scenarios.   For  example, 
most  companies  have  procedures  to  evaluate  credit,  and  it  is  certainly 
realistic  for  a  company  to  have  either  control  over  shipment  authoriza- 
tion or  control  over  physical  shipment  without  necessarily  having  both 
controls.   Another  consideration  in  reducing  the  eight  possible  scenar- 
ios was  the  amount  of  tine  that  would  have  been  required  to  include 
all  of  them  in  the  experiment. 

As  a  result  of  considering  the  above  factors,   it  was  decided 
that  the  inclusion  of  two  scenarios  would  be  the  most  appropriate. 
Therefore,  Scenario  #1  includes  the  existence  of  all  three  key  internal 
controls  and  Scenario  #2  includes  IC-1  (credit  approval)  and  IC-3 
(control  over  physical  shipment).   These  scenarios  provide  the  neces- 
sary information  and  elicit  the  appropriate  responses  for  the  prelimi- 
nary audit  program  stage. 

Within  each  scenario,  there  are  a  given  number  of  "cases." 
These  cases  provide  the  opportunity  to  present  compliance  test  results 
and  elicit  responses  at  the  audit  program  revision  stage.   The  cases 
are  the  possible  combinations  of  compliance  test  results  that  could 
occur  within  a  given  scenario  assuming  that  a  compliance  test  result 
will  either  confirm  or  disconfirm  the  auditor's  prior  belief  about 
the  effectiveness  of  the  control.   For  example,  in  Scenario  #1  where 
all  three  of  the  controls  exist,  there  are  eight  case  situations 
(2x2x2).   in  Scenario  //2,  where  two  of  the  three  controls  exist,  there 

3 
The  factors  were  considered  in  conjunction  with  references  to 
auditing  texts  and  audit  manuals,  along  with  discussions  with  faculty 
members  and  auditors  involved  in  pilot  testing. 
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are  four  case  situations  (2x1x2).   Given  that  no  compliance  tests  are 
performed  if  a  control  is  not  present,  and  that  possible  results  of 
compliance  testing  will  be  either  confirming  or  disconfirming,  the 
experimental  task  at  the  audit  program  revision  stage  consists  of 
twelve  case  situations. 

Decision  process.    The  decision  process  followed  by  the  auditor 
in  this  study  is  summarized  in  flowchart  format  in  Figure  3-3  and  is 
discussed  in  this  section.   The  experimental  task  requires  four 
responses  by  the  auditor  at  various  stages  of  the  audit  process.   First, 
the  auditor  is  asked  the  degree  of  reliability  he  associates  with  the 
existing  internal  controls.   This  response  is  made  after  reviewing 
information  that  is  common  to  all  case  situations,  i.e.,  background 
information  of  the  company,  flowchart  of  the  sales  cycle,  specific 
audit  concern,  etc.,  and  after  being  presented  with  the  combination  of 
key  controls  that  exist  for  a  particular  scenario,  e.g.,  all  three 
controls  exist.   This  first  elicitation  of  reliability  (Rl)  is  required 
for  each  of  the  two  scenarios  and  can  be  represented  as  follows: 
Rl   =   f(Bl,  II,  12,  13), 

where  Rl  =  a  measure  of  reliability  taken  from  a  7-point  scale; 

Bl  =  unchanging  information,  common  to  all  cases; 

11  =  internal  control  #1; 

12  =  internal  control  #2;  and 

13  -  internal  control  #3. 

Recall  that  II,  12,  and  13  are  dichotomous  variables  where  either: 
(i)  the  control  exists  or  (ii)  the  control  does  not  exist. 
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At  this  point,  the  auditor  also  is  asked  for  a  preliminary 
4 
audit  program   (containing  planned  sample  sizes  for  both  compliance 

tests  and  a  substantive  test  relating  to  the  specific  objective). 

After  receiving  specific  results  in  the  form  of  noncompliance  rates 

for  the  compliance  tests,  the  auditor  is  asked  to  indicate  the  degree 

of  reliability  he  associates  with  the  existing  internal  controls. 

This  second  elicitation  of  reliability  (R2)  is  made  for  all  12  cases 

and  can  be  represented  as  follows: 

R2   =   f  (BI,  II,  12,  13,  Rl,  CI,  C2,  C3,  PP), 

where  R2   =   a  measure  of  reliability  taken  from  a  7-point  scale 
(after  observing  results  of  compliance  tests); 

CI   =  result  of  compliance  test  for  II; 

C2   =  result  of  compliance  test  for  12,  if  performed; 

C3  =  result  of  compliance  test  for  13;  and 

PP   =   preliminary  program  as  planned  by  auditor  prior  to 
receiving  results  of  compliance  testing. 

CI,  C2,  and  C3  are  trichotomous  variables  where  either:   (i)  a  compli- 
ance test  was  not  performed  because  the  control  does  not  exist,  (ii) 
the  compliance  test  resulted  in  confirming  evidence,  or  (iii)  the 
compliance  test  resulted  in  disconf irming  evidence. 


4 
In  the  context  of  this  study,  "preliminary  audit  program" 

implies  an  initial  audit  program  that  will  possibly  be  adjusted  for 

the  substantive  test  as  a  result  of  the  evaluation  of  compliance  test 

results.   At  that  time  a  "final  audit  program"  is  determined. 

The  confirmation  of  accounts  receivable  was  chosen  for  use  in 
this  study  because  of  its  most  direct  relationship  with  the  sales 
account  and  its  familiarity  to  the  subjects. 
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The  fourth  elicitation,  a  final  sample  size  for  the  substantive 
test  (FS),  also  is  required  for  all  12  cases  and  can  be  represented  as 

follows: 

FS   =  f(BI,  II,  12,  13,  Rl,  CI,  C2,  C3,  PP,  R2). 

Experimental  materials.    The  materials  provided  to  the  sub- 
jects are  presented  in  Appendix  A  and  included  the  following: 

(1)  a  set  of  instructions  to  explain  the  purpose  of  the  study, 
the  setting  in  which  the  subject  would  operate,  the 
factors  that  would  be  varied,  and  the  specific  decisions 
that  were  to  be  required  of  each  subject; 

(2)  background  information  to  describe  the  company  to  be 
audited  and  additional  information  that  was  gathered 
during  preliminary  audit  work  (i.e.,  the  existence  of 
the  key  internal  controls  and  other  controls  that  may 
impact  on  the  attainment  of  the  specific  objective); 

(3)  a  flowchart  representation  of  the  sales  cycle  of  the 
hypothetical  company,  with  representations  of  the  three 
key  internal  controls; 

(4)  a  statement  of  the  specific  audit  objective  to  be  con- 
sidered and  a  summary  description  of  the  controls  of 
interest  and  the  corresponding  preliminary  audit  program 
(containing  procedures  for  both  compliance  testing  and 
the  appropriate  substantive  test); 

(5)  specific  information  relating  to  each  of  the  12  individual 
cases  (this  includes,  for  example,  the  extent  to  which  the 
three  key  internal  controls  exist  and  the  results  from 
performing  the  desired  amount  of  compliance  testing); 

(6)  tables  for  determining  the  sample  sizes  for  both  compli- 
ance tests  and  the  substantive  test  using  a  statistical 
sampling  approach,  along  with  tables  for  interpreting 
the  results  of  compliance  tests  which  were  presented  in 
the  form  of  noncompliance  rates  for  specific  attributes; 
and 

(7)  a  post-experiment  questionnaire. 


49 


Cue  Selection 

The  decision  to  present  an  audit  situation  with  a  specific 
objective  was  based  primarily  on  the  desire  to  maximize  control  over 
cues  provided  to  the  subjects.   The  audit  environment  is  recognized  as 
very  complex  with  many  interrelated  components.   To  say  that  the  three 
key  controls  highlighted  in  this  experiment,  in  conjunction  with  the 
accounts  receivable  confirmation  results,  are  the  only  important  cues 
in  deciding  whether  "recorded  sales  are  for  valid  transactions"  is  of 
course  not  true.   However,  as  a  result  of  reviewing  auditing  textbooks, 
audit  manuals  from  "Big-Eight"  accounting  firms,  and  discussing  this 
particular  audit  situation  with  those  auditors  participating  in  the 
pilot  studies,  the  three  key  controls  were  considered  most  appropriate 
in  deciding  the  number  of  accounts  receivable  confirmations  to  send 
relative  to  this  particular  objective. 

Other  information  that  could  be  deemed  important  in  meeting  the 
specific  objective  was  provided.   It  was  indicated  to  subjects  that  the 
review  procedures  revealed  no  problems  relating  to  monthly  statements, 
customer  complaints,  and  prenumbering  documents.   It  also  was  stated 
that  normal  review  procedures  would  be  followed  in  such  related  areas 
as  inventory  cut-off,  review  of  aged  receivables,  uncollectibles,  etc. 
Appendix  A  includes  the  information  given  the  participants  concerning 
these  areas  and  the  primary  cues  to  be  used  in  determining  the  reli- 
ability of  the  system  and  the  resulting  extent  of  substantive  testing 
for  this  particular  audit  objective. 
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Experimental  Design 

The  experiment  was  conducted  using  a  2x3x2  factorial  design 
(fixed  effects  model)   and  was  performed  by  109  subjects.   A  fac- 
torial design  offers  advantages  in  the  evaluation  of  the  cues  (factors) 
used  in  the  experiment.   Specifically,  factorial  designs  allow  for  the 
evaluation  of  a  combined  effect  of  two  or  more  variables  (factors) 
when  used  simultaneously.   The  more  complete  information  that  results 
has  been  described  by  Winer: 

Information  obtained  from  factorial  experiments  is 
more  complete  than  that  obtained  from  a  series  of 
single  factor  experiments,  in  the  sense  that  factorial 
experiments  permit  the  evaluation  of  interaction 
effects.   An  interaction  effect  is  an  effect  attribut- 
able to  the  combination  of  variables  above  and  beyond 
that  which  can  be  predicted  from  the  variables  con- 
sidered singly  [1971,  p.  309]. 

The  impact  of  interaction  effects  will  vary  from  study  to  study.   How- 
ever, it  is  unusual  for  any  higher  order  interactions  greater  than 
those  of  two  factors  to  contribute  significantly  to  the  model  (e.g., 
Goldberg,  1968).   Accordingly,  the  focus  of  this  experiment  upon  main 
effects  and  two-factor  interactions  should  not  constitute  a  serious 
limitation. 

As  discussed  previously,  two  broad  scenarios  were  presented  to 
the  subjects.  Scenario  //l  indicates  that  all  three  controls  (factors) 
exist  and  each  control  has  dichotomous  values,  either:   1)  a 


See  Winer  [1971,  p.  312]  for  a  discussion  of  the  fixed  effects 
(factors)  model. 

There  were  a  total  of  119  packages  distributed.   Ten  subjects 
were  not  included  in  the  analysis  because  they  submitted  incomplete 
packages  or  responses  that  indicated  a  lack  of  understanding  of  the 
requirements  of  the  study. 
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2%  noncompliance  rate  that  implies  confirming  evidence,  or  2)  an  8%9 
noncompliance  rate  that  implies  disconf inning  evidence. 

Scenario  #2  presents  the  situation  where  IC-2  does  not  exist 
and  therefore  will  have  no  compliance  testing  performed.   This  allows 
for  a  third  value  (level)  for  IC-2  compared  to  only  two  values  for  IC-1 
and  IC-3.   Therefore,  the  experiment  has  three  factors  (IC-1,  IC-2, 
and  IC-3)  with  two  possible  levels  for  IC-1  and  IC-3  and  three  possible 
levels  for  IC-2.   This  results  in  the  2x3x2  factorial  design  with  the 
three  factors  and  their  levels  as  shown  in  Table  3-1. 

The  design  allows  for  the  estimation  of  all  main  effects  and 
two  factor  interactions  with  the  11  degrees  of  freedom  available.   The 
degrees  of  freedom  for  individual  analysis  are  used  as  follows: 

effect  d.f . 

main  4 

two  factor  interaction       5 
higher-order  (error)         2 

total  11 

As  indicated,  the  mean  square  from  the  higher  order  interactions  was 
combined  to  form  the  error  term. 

The  12  cases,  representing  the  12  cells  of  the  factorial 
design,  were  presented  in  a  random  order  within  each  scenario,  with 
Scenario  #1  always  presented  first.   Approximately  one-third  of  the 


This  percentage  was  given  as  the  "expected"  noncompliance 
rate. 

9 
This  percentage  was  developed  through  discussions  with 
auditors  and  references  to  other  audit  situations.   It  provides  an 
achieved  precision  level  that  is  obviously  unacceptable  and  therefore 
represents  "unexpected"  or  "disconf inning"  evidence. 
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TABLE  3-1 
FACTORS  AND  LEVELS  USED  IN  THE  STUDY 


Factors 


1.   IC-1:   Credit  Approval 


Results  of 
Level      Compliance  Tests 


2.   IC-2:   Shipment  Authoritation 


1  2%  noncompliance  rate 

2  8%  noncompliance  rate 


3.   IC-3:   Physical  Shipment 


1  2%  compliance  rate 

2  8%  noncompliance  rate 

3  Not  performed 

1  2%  noncompliance  rate 

2  8%  noncompliance  rate 
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subjects  reviewed  the  cases  in  one  randomized  order  and  the  remaining 
two-thirds  in  a  different  randomized  order. 

Description  of  the  Auditor's  Judgment  Process 

Univariate  analysis  of  variance  (ANOVA)  was  applied  to  the 
reliability  and  sample  size  responses  for  the  12  factor  combinations 
of  each  subject.   Also,  ANOVA  was  applied  to  the  responses  by  groups 
according  to  firm  and  experience  levels  of  the  subjects. 

F  ratios  and  levels  of  significance  were  computed  for  each  main 
effect  and  two-factor  interaction.   As  discussed  previously,  significant 
interaction  effects  are  indicative  of  configural  processing  and  are  not 
predominant  in  human  information  processing  research  findings. 

The  omega-squared  statistic  was  calculated  for  both  reliability 
and  sample  size  responses  to  determine  the  proportion  of  variance 
accounted  for  by  each  main  effect  and  two-factor  interaction.   The 
omega-squared  value  indicates  the  proportion  of  total  variance  accounted 
for  by  a  particular  factor. 

Judgment  Consensus 

Of  major  interest  in  this  study  is  the  degree  of  consensus  or 
agreement  among  subjects  when  responding  to  the  same  data.   The  degree 
of  consensus  was  evaluated  at  the  audit  program  revision  stage  using 
Pearson  product-moment  correlation,  canonical  correlation,  and  cluster 
analysis. 


This  imbalance  resulted  from  the  unexpectedly  large  number 
of  participants  provided  by  one  of  the  firms. 
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Pearson  correlation  measures  were  calculated  among  subjects  on 
their  reliability  decisions  and  then  on  the  sample  size  decisions  for 
the  12  cases.   This  resulted  in  two  measures  of  the  strength  of  the 
linear  relationship  of  the  responses  between  each  pair  of  subjects. 

Canonical  correlation  analysis  can  consider  the  reliability 
and  sample  size  resonses  simultaneously,  enabling  the  derivation  of  a 
linear  combination  for  each  subject  such  that  the  correlation  between 
each  pair  of  subjects  is  maximized.   This  correlation  measure  is  the 
canonical  correlation  developed  and  discussed  for  the  5,886  possible 
combinations  of  subjects  in  this  study. 

The  cluster  analysis  was  performed  both  on  the  reliability  and 
sample  size  responses  individually  and  combined.   The  cluster  program 
used  a  hierarchical  clustering  technique  [Johnson,  1967]  which  begins 
by  forming  a  cluster  for  each  subject  in  the  analysis.   Using  a  Euclid- 
ean distance  measure,  the  two  closest  clusters  then  are  combined  into 
one  cluster,  the  two  closest  of  the  new  set  then  are  combined  into  a 
single  cluster,  and  so  on. 

Self-Insight 

Another  area  of  relative  importance  in  this  study  is  the 
extent  of  self-insight  possessed  by  the  subjects.   That  is,  does  the 
auditor  have  a  good  "feel"  for  the  judgment  process  he  employed  in 
arriving  at  his  decisions?   This  question  specifically  addresses  the 
auditor's  ability  to  understand  and  express  the  relative  weights  or 
importance  of  the  cues  used  to  arrive  at  his  decisions. 
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To  allow  for  the  calculation  of  a  self-insight  index,  a  post- 
experiment  questionnaire  asked  each  subject  to  allocate  100  points  to 
the  three  key  internal  controls  so  as  to  reflect  the  relative  importance 
of  each  to  his  decisions.   A  self-insight  index  was  calculated  following 
the  approach  employed  by  Slovic  [1969],  Ashton  [1973],  and  Messier 
[1979].   The  subjective  weights  elicited  from  the  subjects  were  corre- 
lated with  adjusted  objective  weights  calculated  from  the  omega-squared 
values  (discussed  earlier).   The  omega-squared  values  were  adjusted  by 
normalizing  the  values  to  100  for  the  main  effects.   The  original 
omega-squared  value  for  a  main  effect  was  increased  by  the  omega- 
squared  values  for  any  interaction  terms  containing  that  particular  main 
effect.   For  example,  the  omega-squared  values  for  the  interaction  terms 
(IC-1) (IC-2)  and  (IC-1)  (IC-3)  were  added  to  the  omega-squared  value  for 
the  main  effect  of  IC-1.   The  adjusted  omega-squared  values  for  the 
three  internal  controls  were  summed  and  then  each  value  was  divided  by 
that  sum.   The  results  are  the  adjusted  objective  weights  for  the  three 
internal  controls  which  sum  to  100  and  can  be  correlated  with  the  sub- 
jective weights  given  by  the  subjects  to  provide  a  self-insight  index 
for  each  subject. 

Lack  of  Formal  Hypotheses 

The  overall  objective  of  this  study  as  given  previously  is  to 
describe  and  analyze  the  auditor's  judgment  as  reflected  in  a  routine 
audit  situation.   It  should  be  stressed  that  due  to  the  emphasis  on 
the  descriptive  nature  of  this  study  there  are  no  formal  hypotheses  to 
be  tested.   However,  previous  studies  concerning  auditor  judgment 
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provide  some  basis  for  expectations.   Accordingly,  expected  results  of 
the  current  study  are  discussed  below. 

The  extent  of  auditor  consensus  relative  to  some  recent  judgment 
studies  was  discussed  in  Chapter  II.   The  current  study  is  characterized 
by  a  similar  audit  setting  requiring  similar  decisions.   Evidence  con- 
cerning consensus  is  provided  at  the  preliminary  audit  program  stage, 
with  more  extensive  evidence  provided  at  the  audit  program  revision 
stage.   Previous  studies  have  indicated  various  degrees  of  consensus 
among  auditors.   Due  to  the  nature  of  the  audit  setting  and  the  audit 
task,  some  lack  of  consensus  is  expected  in  this  study.   However,  the 
existence  of  some  consensus  is  more  likely  to  appear  as  the  auditors 
are  grouped  according  to  firm  affiliation  or  experience  level. 

More  specifically,  it  is  anticipated  that  moving  from  level  1 
(confirming  evidence)  to  level  2  (disconf irming  evidence)  will  cause 
the  degree  of  reliability  of  the  controls  to  decrease  and  the  sample 
size  for  the  substantive  test  to  increase.   However,  the  movement  to 
level  3  (control  does  not  exist)  from  level  2  for  factor  2  will  not 
necessarily  have  the  same  effect.   The  question  becomes:   Which  situa- 
tion is  considered  "stronger,"  to  have  a  control  that  is  ineffective  in 
terms  of  allowing  an  unacceptable  noncompliance  rate  or  for  the  control 
not  to  exist?   The  control  may  not  exist  because  it  is  not  needed.   For 
this  reason,  it  is  expected  that  there  will  be  a  lack  of  consensus  on 
this  point. 

With  regard  to  the  decision  models  developed  from  the  auditors' 
responses,  it  is  expected  that  the  main  effect  variables  will  account 
for  most  of  the  variance  when  combined  in  a  linear  fashion.   Two-factor 
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interactions  are  expected  to  account  for  some  variance,  with  higher- 
order  interactions  being  insignificant. 

Disagreement  among  the  auditors  is  expected  with  respect  to 
the  relative  importance  of  the  three  controls  (variables).   An  a  priori 
subjective  ranking  of  the  relative  importance  of  the  controls  within 
the  specific  setting  of  this  study  is  possible,  but  was  not  attempted 
because  of  the  subjective  nature  of  such  a  ranking.   whatever  the 
relative  importance  of  the  variables  attached  by  the  auditors,  pre- 
vious accounting  studies  have  indicated  a  high  degree  of  self-insight 
by  the  participants.   A  similar  degree  of  self-insight  is  expected  in 
this  study. 

Administration  of  the  Experiment 
Pilot  Studies 

Several  pilot  studies  were  conducted  to  provide  essential  feed- 
back on  the  experimental  materials.   Two  initial  sessions  were  conducted 
with  individual  auditors  who  went  through  the  experiment,  followed  by  a 
discussion  of  their  comments  and  suggestions. 

The  first  participant  was  a  member  of  a  local  CPA  firm  who  had 
previously  been  with  a  "Big-Eight"  firm  and  has  a  specific  interest  in 
statistical  sampling.   His  comments  dealt  mostly  with  clarifications 
with  respect  to  the  representations  on  the  flowchart  and  the  use  of  the 
statistical  tables  provided  for  the  various  decisions. 

The  second  participant  was  chosen  primarily  because  of  his 
affiliation  with  one  of  the  firms  that  would  be  participating  in  the 
primary  study.   His  experience  level  and  training  also  were  comparable 
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to  those  participants  in  the  primary  study.   Some  changes  as  to  wording 
and  clarity  were  made  as  a  result  of  his  comments.   However,  his  major 
contribution  was  that  he  felt  members  of  his  firm  at  his  experience 
level  would  feel  "comfortable"  with  the  audit  situation  presented  by 
the  experimental  materials. 

The  final  pilot  study  was  conducted  in  early  August,  1979,  with 
three  members  of  one  of  the  primary  study  firms.   Their  major  concern 
was  that  other  audit  procedures  important  in  this  experimental  setting 
must  be  accounted  for  in  some  manner.   Their  specific  recommendations 
in  this  area  along  with  some  additional  changes  in  wording  were  incor- 
porated into  the  materials. 

A  final  revision  in  the  experimental  materials  was  made  as  the 
result  of  discussion  with  a  "Big-Eight"  partner  with  a  strong  research 
background.  His  comments  led  to  some  key  changes  in  the  wording  of 
the  specific  objective,  a  change  in  one  of  the  key  controls,  and  other 
important  changes  necessary  for  additional  "realism"  and  clarity.  The 
basic  structure  and  organization  of  the  experimental  materials  and  the 
response  modes,  however,  remained  unchanged  from  earlier  pilot  studies. 

Primary  Study 

The  primary  study  was  conducted  with  four  public  accounting 
firms  in  August  and  September,  1979.   The  study  was  conducted  in  the 
offices  of  three  of  the  participating  firms  and  in  a  conference  room 
of  a  hotel  where  a  seminar  was  being  conducted  by  the  other  firm.   The 
experimenter  was  in  charge  and  present  for  the  duration  of  all  adminis- 
trations of  the  experiment.   The  same  set  of  instructions  were  read  to 
all  participants  with  no  additional  comments  made  or  questions  answered 
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during  the  approximately  70  minutes  of  each  administration.  Although 
there  was  no  time  limit  given,  participants  were  told  to  expect  to 
spend  50-60  minutes  going  through  the  experiment  and  actually  spent 
anywhere  from  30  to  75  minutes  with  a  mean  of  about  55  minutes. 

The  four  firms  that  participated  were  all  members  of  the  "Big- 
Eight."   Three  of  the  firms  provided  participants  from  a  single  office 
in  the  same  city.   The  other  firm  provided  subjects  from  various 
offices  who  were  brought  together  through  their  participation  in  a 
statistical  sampling  seminar.   The  experience  levels  of  those  partici- 
pating varied  somewhat  from  firm  to  firm  and  are  summarized  in  Table 
3-2  according  to  the  experience  classifications  used  in  Chapter  IV  to 
analyze  the  results  of  the  study.   Also,  the  number  of  participants 
from  each  firm  varied  somewhat  due  to  the  fact  that  one  criterion  for 
choosing  subjects  was  their  availability.   The  major  difference  was 
with  respect  to  Firm  //4,  where  75  usable  responses  were  obtained  as 
compared  to  12,  10,  and  12  for  Firms  1-3  respectively. 

Limitations  of  the  Experiment 

There  are  certain  inherent  limitations  associated  with  behav- 
ioral studies.   In  addition  to  the  problems  with  the  development  and 
conduction  of  the  experiment  itself,  other  shortcomings  arise  relative 
to  the  analysis  of  the  information  generated.   Specific  limitations  of 
this  study  are  discussed  below  and  include: 

1.  the  selection  of  subjects; 

2.  the  audit  environment; 

3.  the  administration  of  the  experiment; 

4.  the  ability  to  explain  the  judgment  process;  and 

5.  the  generalizability  of  results. 
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TABLE  3-2 
EXPERIENCE  LEVEL  OF  SUBJECTS  BY  FIRM 

Experience  Level 

E-l  (1  or  2  yrs)     E-2  (>  3  yrs) 


Firm  #1 


Firm  #2 


Firm     Firm  #3 


Firm  #4 


Total 


Total 


— 

12 

12 

2 

8 

10 

11 

1 

12 

39 

34 

73 

52 

55 

107 

Firm  #1 


Firm  #2 


Firm     Firm  #3 


Firm  #4 


Total 


Experience  Level 
X-l  (1-3  yrs)      X-2  (>  4  yrs)     Total 


3 

9 

12 

6 

4 

10 

12 

— 

12 

64 

9 

73 

85 

22 

107 

NOTE:   Subjects  100-4  and  102-4  could  not  be  classified  as  to  experi- 
ence and  are  not  included  in  this  table. 
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Selection  of  Subjects 

Studies  which  require  the  use  of  auditors  are  restricted  by 
the  availability  of  participants.   To  assemble  a  group  of  auditors 
together  for  the  sake  of  performing  a  task  such  as  that  required  in 
this  study  constitutes  a  substantial  intrusion  upon  their  time  and 
therefore  is  costly  to  their  firms.   For  this  reason,  availability 
becomes  a  major  consideration  in  determining  the  participants  from 
each  firm.   Also,  those  responsible  for  the  selection  of  participants 
from  each  firm  may  want  their  "best"  people  to  represent  them  in  such 
a  study.   On  the  other  hand,  the  "best"  people  may  be  out  on  assign- 
ment while  the  others  participate  in  the  "academic  exercise"  in  the 
office.   Whatever  the  method  of  selection  of  participants,  it  must 
be  recognized  as  "non-random"  and  in  some  cases  possibly  biased. 
Whether  the  results  of  such  a  sample  represent  the  cross-section  of 
senior  auditors  from  "Big-Eight"  accounting  firms  is  open  for  dis- 
cussion.  This  limitation  is  therefore  noted  with  the  belief  that  the 
impact  of  such  a  shortcoming  on  implications  of  this  study  is  negli- 
gible. 

Audit  Environment 

The  experimental  materials  were  developed  with  the  goal  of 
presenting  the  auditors  with  a  realistic  setting  in  which  they  would 
have  little  difficulty  responding  to  the  questions  presented.   A 
"self-contained"  audit  situation  was  sought  to  the  extent  possible  in 
order  to  better  control  for  extraneous  variables.   However,  this 
results  in  the  sacrifice  of  a  certain  amount  of  realism  due  to  the 
dynamic  and  inter-related  nature  of  the  audit  process. 
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The  resulting  audit  situation  is  admittedly  lacking  in  some 
"realism"  characteristics.   It  was  felt  that  any  further  attempts  at 
realism  would  have  introduced  confounding  factors  that  serve  to 
reduce  the  internal  validity  of  the  experiment.   The  problem  associ- 
ated with  creating  a  realistic  audit  setting  is  discussed  in  the 
following  section. 

Administration  of  the  Experiment 

Although  the  experiment  was  conducted  in  a  controlled  setting 
by  the  experimenter,  some  problems  are  still  recognized.   Whether  an 
auditor  will  make  "meaningful"  audit  decisions  in  a  "contrived"  audit 
situation  with  no  reward  or  penalty  structure  must  always  be  of  concern. 
However,  positive  feedback  was  received  on  the  materials  and  the  con- 
duct of  the  experiment  with  no  apparent  motivational  problems  among 
those  participating. 

The  situation  in  which  seventy-five  participants  from  one  firm 
performed  the  study  in  a  single  room  posed  some  administrative  problems. 
More  participants  were  given  the  same  order  of  cases  as  a  result  of 
duplication  problems  and  there  was  not  the  total  lack  of  communication 
with  others  that  was  assured  with  the  small  groups  in  the  other  firms. 
These  shortcomings  did  not  seem  to  create  any  significant  problems  in 
the  current  study. 

Explaining  the  Judgment  Process 

To  explain  the  judgment  process  is  a  most  admirable  research 
goal.  This  section  serves  as  a  reminder  that  it  was  not  the  purpose 
of  this  study  to  explain  the  judgment  process  of  auditors  within  the 
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context  of  internal  control  evaluation.   There  is  no  attempt  to  infer 
any  understanding  of  the  judgment  process  through  the  evaluation  of 
any  statistical  techniques  such  as  ANOVA  or  MANOVA.   However,  the 
results  of  the  application  of  ANOVA  procedures  are  presented  as  part 
of  the  results  of  this  study  in  an  attempt  to  describe  the  judgment 
process. 

General izability  of  Results 

A  non-randomly  chosen  group  of  auditors  was  presented  a 
specific  audit  situation  and  asked  to  evaluate  a  hypothetical  company 
and  respond  to  a  set  of  questions  regarding  the  extent  of  audit  work 
to  perform.   The  participants  spent  approximately  sixty  minutes  in 
the  experiment  with  no  anticipation  of  reward  or  fear  of  penalty. 
For  these  reasons,  generalizing  from  the  results  of  this  study  to 
other  audit  situations  for  these  auditors  or  to  other  auditors  in  the 
same  audit  situation  is  not  justified. 

The  inability  to  generalize  the  results  of  this  study  does  not 
distract  from  the  specific  benefits  derived  and  the  implications  for 
future  research  that  will  be  discussed  in  Chapter  V 

Summary 

This  chapter  has  introduced  the  framework  in  which  the  current 
study  was  conducted   The  lens  model  was  presented  along  with  ANOVA  as 
most  appropriate  for  the  analysis  of  the  judgment  process  of  auditors. 
The  dependence  upon  correlational  measures  was  also  emphasized. 
Within  this  described  framework,  the  experimental  task  was  discussed, 
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both  as  to  support  for  its  selection  and  to  the  specific  experimental 
materials. 

The  selection  of  a  2x3x2  factorial  design  was  discussed  in 
conjunction  with  the  various  possible  levels  for  each  factor  (cue). 
The  analysis  techniques  that  were  used  within  the  described  framework 
and  to  fulfill  the  objectives  of  this  study  were  presented.   These 
included  the  use  of  correlation  (both  univariate  and  multivariate), 
ANOVA  (or  MANOVA) ,  cluster  analysis,  and  "self-insight"  indices  (as 
developed  from  omega-squared  values). 

Without  the  statement  of  formal  hypotheses,  some  expected 
results  of  this  study  were  presented.   While  some  of  these  results 
were  inferred  from  reference  to  similar  previous  studies,  others  were 
inferred  from  an  evaluation  of  the  professional  literature  and  the 
practice  of  auditing.   The  procedures  followed  in  both  the  pilot  studies 
and  primary  study  were  presented  along  with  information  concerning  the 
participants  involved. 

Finally,  certain  limitations  of  the  experiment  were  presented 
and  discussed.   None  of  the  limitations  are  expected  to  have  a  signifi- 
cant impact  on  the  results  of  the  study  which  are  presented  in  Chapter 
IV. 


CHAPTER  IV 
RESULTS  OF  THE  STUDY 

Introduction 

The  results  of  the  study  are  presented  and  discussed  in  this 
chapter.   As  noted  in  Chapter  III,  decisions  were  required  by  the 
auditors  at  both  the  preliminary  audit  program  stage  and  at  the  audit 
program  revision  stage.   Due  to  the  structure  of  the  study,  the  analy- 
sis of  responses  from  the  audit  program  revision  stage  is  emphasized. 
This  chapter  first  will  present  a  discussion  of  the  responses  relating 
to  the  formulation  of  a  preliminary  audit  program.   From  examining  these 
preliminary  decisions,  some  comments  are  made  relating  to  a  comparison 
of  the  participating  firms  and  to  the  relative  importance  of  the  three 
IACs  used  in  the  study.   Finally,  and  in  much  greater  detail,  an  analy- 
sis of  the  responses  from  the  audit  program  revision  stage  is  presented. 

The  additional  data  gathered  from  the  twelve  case  situations 
presented  at  the  audit  program  revision  stage  allow  for  a  more  extensive 
analysis.   First,  results  of  the  descriptive  analysis  of  the  judgment 
process  of  the  auditors  required  in  their  IAC  reliability  and  substan- 
tive test  sample  size  decisions  are  presented.   Second,  the  extent  of 
judgment  consensus  is  evaluated.   Finally,  the  extent  of  self-insight 
exhibited  by  the  auditors  is  discussed.   Although  some  interpretations 
of  the  results  in  these  areas  are  presented  within  this  chapter,  the  pri- 
mary evaluation  of  results  and  discussion  of  implications  are  provided 
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in  Chapter  V.   The  final  sections  of  the  present  chapter  present  summa- 
ries of  additional  data  gathered  in  the  post-experiment  questionnaire. 

Preliminary  Audit  Program 

For  the  preliminary  audit  program  stage,  the  auditors  in  the 
study  responded  to  questions  on:   (1)  the  reliability  of  the  IAC  system 
based  only  on  the  knowledge  of  the  existence  (nonexistence)  of  the  three 
key  internal  controls,  (2)  the  desired  sample  sizes  for  the  compliance 
tests  to  be  performed  on  the  existing  controls,  and  (3)  the  preliminary 
sample  size  for  the  substantive  test.   The  limited  amount  of  data  gath- 
ered from  these  questions  restricts  the  analysis  both  within  this  stage 
of  the  audit  process  and  the  comparison  with  the  data  gathered  at  the 
audit  program  revision  stage.   However,  some  interesting  relationships 
can  be  established  from  examining  responses  from  the  preliminary  audit 
program  stage. 

Summaries  of  responses  for  the  preliminary  audit  program  are  pre- 
sented in  Table  4-1  and  Table  4-2.   Table  4-1  provides  the  mean  responses 
by  firm  for  the  reliability  and  sample  size  (compliance  and  substantive 
test)  decisions.   Using  the  data  gathered  at  this  point  in  the  audit 
process,  some  specific  observations  can  be  made  regarding  the  partici- 
pants and  the  three  key  internal  accounting  controls  used  in  this  study. 

Comparison  Among  Firms 

The  preliminary  responses  shown  in  Table  4-1  provide  a  basis 
for  considering  the  responses  of  Firm  #1  and  Firm  #4  to  be  similar, 
while  the  auditors'  resonses  from  Firm  #2  and  Firm  #3  could  also  be 
considered  similar  to  one  another.   In  addition,  the  preliminary 
responses  to  the  substantive  test  sample  size  questions  are  the  basis 
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TABLE  4-1 

MEAN  RESPONSES  BY  FIRM  FOR 
THE  PRELIMINARY  AUDIT  PROGRAM 


SCENARIO  #1  -  Existence  of  IC-1,  IC-2,  and  IC-3 


Firm  #1    Firm  //2    Firm  #3    Firm  #4    Combined 


Reliability 

5.00 

5.80 

5.00 

5.11 

5.15 

CT:   IC-1 

77.08 

122.50 

106.25 

80.67 

86.93 

CT:   IC-2 

93.75 

137.50 

129.17 

99.67 

105.73 

CT:   IC-3 

118.75 

145.00 

179.17 

107.93 

120.37 

Sub.  Test 

63.33 

84.10 

74.67 

46.83 

55.13 

SCENARIO  #2  -  Existence  of  IC-1  and  IC-3 


Firm  //l 

Firm  ill 

Firm  //3 

Firm  #4 

Combined 

Reliability 

4.33 

4.30 

4.27 

4.06 

4.14 

CT:   IC-1 

77.08 

97.50 

143.75 

88.85 

94.44 

CT:   IC-3 

125.00 

140.00 

181.25 

128.37 

134.94 

Sub.  Test 

78.83 

130.20 

110.08 

69.81 

80.78 
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for  considering  Firm  #2  and  Firm  //3  to  be  more  conservative  than  Firm 
//l  and  Firm  #4.  . 

The  preliminary  reliability  responses  for  the  two  scenarios 
are  as  expected  in  that  the  reliability  level  decreased  as  IC-2  was 
eliminated  as  an  existing  control.   On  the  average,  when  moving  to 
Scenario  #2,  the  reliability  level  decreased  by  1.1  (on  the  7-point 
scale);  however,  for  Firm  #2  the  reliability  level  decreased  by  1.5. 
Possible  explanations  are  that  either  Firm  #2  is  more  conservative 
than  the  other  firms,  or  that  the  auditors  from  Firm  #2  perceived  IC-2 
to  be  more  important  than  the  auditors  from  the  other  firms.   Evidence 
of  greater  importance  being  attributed  to  IC-2  is  provided  by  the  sub- 
sequent analysis  of  the  responses  of  the  auditors  (i.e.,  see  Table  4-2 
and  Table  4-15).   In  addition,  indications  of  the  conservativeness  of 
Firm  #2  are  found  in  the  auditors'  responses  to  the  preliminary  deci- 
sion as  to  the  sample  size  for  the  substantive  test.   For  Scenario  #1, 
auditors  from  Firm  #2  suggested  a  preliminary  sample  size  for  the  sub- 
stantive test  of  84.10,  as  compared  to  a  mean  response  from  the  auditors 
of  the  other  firms  of  52.20.   Similarly,  when  responding  to  the  same 
question  for  Scenario  #2,  auditors  from  Firm  //2  had  a  mean  response  of 
130.20  as  compared  to  a  mean  response  of  75.78  for  all  other  auditors. 

The  above  discussion  provides  evidence  of  some  differences 
among  the  firms  at  the  preliminary  audit  program  stage.   While  some 
differences  exist  among  the  firms  in  the  selection  of  sample  sizes  for 
the  compliance  and  substantive  tests,  an  indication  of  agreement  as  to 
the  relative  importance  of  the  three  IACs  is  evident. 
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Relative  Importance  of  IACs 

Table  4-2  summarizes  the  number  of  auditors  who  used  each  of 
the  various  confidence  levels  in  testing  the  three  key  IACs.   Generally, 
controls  that  are  felt  to  be  of  greater  importance  are  tested  at  a 
higher  confidence  level  than  those  controls  of  lesser  importance. 
Therefore,  IC-3  (control  over  physical  shipment)  appears  to  be  the  most 
important  control  across  all  firms.   Although  members  of  Firm  #2  indi- 
cated the  same  selection  of  confidence  levels  for  IC-1  and  IC-3  for 
Scenario  #1,  the  selections  for  Scenario  #2  indicated  slightly  higher 
confidence  levels  for  IC-3.   IC-2  (shipment  authorization)  seems  to 
follow  IC-3  in  importance,  with  IC-1  (credit  approval)  considered  the 
least  important.    As  following  sections  will  indicate,  these  responses 
are  consistent  with  the  more  extensively  analyzed  responses  at  the 
audit  program  revision  stage. 

Audit  Program  Revision 

The  objectives  of  this  study  are  pursued  primarily  through  the 
examination  of  the  responses  at  the  audit  program  revision  stage.   The 
twelve  case  situations  presented  to  the  auditors  included  the  results 
of  compliance  testing.   These  tests  assumed  the  use  of  the  suggested 
sample  sizes  that  were  elicited  in  the  preliminary  audit  program.   Due 
to  the  number  of  cases,  the  data  generated  from  the  reliability  and 
substantive  test  sample  size  responses  allow  for  a  more  extensive 
evaluation  and  analysis  than  that  from  the  preliminary  audit  program 
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Any  statements  concerning  the  relative  importance  of  these 
three  controls  are  made  only  within  the  context  of  the  current  study 
and  should  not  be  considered  generalizable. 
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stage.   The  analysis  that  follows  will  include  an  examination  of 
responses  made  by  the  individual  auditors  as  well  as  a  comparison  of 
responses  among  the  firms  and  the  various  experience  levels. 

Description  of  the  Reliability/Sample  Size  Judgment 

The  experimental  cases  were  arranged  in  a  factorial  design 
(2x3x2)  which  permitted  analysis-of-variance  (ANOVA)  computations  for 
each  auditor  participating  in  the  study.   ANOVA  was  used  to  evaluate 
the  relative  importance  of  the  key  internal  controls  provided  for  this 
specific  reliabilty/sample  size  judgment. 

The  ANOVA  analysis  was  performed  individually  for  both  the 
reliability  decisions  and  the  sample  size  decisions.   Therefore,  an 
ANOVA  model  was  constructed  for  each  subject  corresponding  to  their 
reliability  decisions  and  to  their  sample  size  decisions.   The  omega- 
squared  values  for  each  main  effect  and  two-factor  interaction  are 

2 
presented  by  subject  (auditor)   in  Table  4-3  (reliability  judgment) 

3 
and  Table  4-4  (sample  size  judgment).    It  will  be  recalled  that  the 

omega-squared  value  indicates  the  proportion  of  total  variance 

accounted  for  by  a  particular  IAC  or  combination  of  controls. 

As  is  indicated  in  Tiibles  4-3  and  4-4,  tests  for  significant 

factors  could  not  be  conducted  for  seven  auditors  with  respect  to  the 


2 
The  auditors  are  numbered  from  1-1  to  109-4,  with  the  second 
number  representing  the  firm  affiliation  of  the  auditor. 

3 
ANOVA  was  originally  conducted  using  the  3-vay  interaction  to 
form  the  error  term.   With  a  mean-square-error  of  zero  for  24  and  19 
auditors  on  the  reliability  and  sample  size  judgments,  respectively,  a 
second  ANOVA  was  conducted  for  these  auditors  using  2-way  and  3-way 
interactions  to  form  the  error  term.   The  omega-squared  calculations 
resulting  from  this  second  ANOVA  are  noted  with  an  asterisk  (*)  in 
Tables  4-3  and  4-4,  and  are  included  in  all  further  anlyses  unless 
otherwise  noted. 
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reliability  judgments  and  for  eight  auditors  with  respect  to  the  sample 
size  judgments.   An  examination  of  the  responses  of  these  auditors 
revealed  a  lack  of  variability  in  their  responses.   In  fourteen  of  the 
fifteen  cases,  the  auditor's  responses  fluctuated  between  only  two 
numbers.   For  example,  auditor  number  10  from  Firm  //l  responded  with 
either  a  4  or  2  for  the  reliability  judgment  and  either  80  or  107  for 
the  sample  size  judgment.   The  following  discussion  of  individual  ANOVA 
results  is  based  on  the  102  usable  results  from  the  reliability  judg- 
ment and  101  usable  results  from  the  sample  size  judgments. 

Results  from  the  individual  ANOVAs  (Tables  4-3  and  4-4)  indi- 
cate that  IC-3  (control  over  physical  shipment)  accounted  for  most  of 
the  variance  in  their  judgments.   Specifically,  65  of  102  auditors  for 
the  reliability  judgment  and  57  of  101  for  the  sample  size  judgment 
indicated  greatest  reliance  on  IC-3.   Auditors  whose  responses  indi- 
cated IC-2  (shipment  authorization)  accounted  for  most  of  the  variance 
numbered  21  and  28  for  reliability  and  sample  size,  respectively, 
whereas  the  result  for  IC-1  (credit  approval)  indicated  13  and  10 
auditors  for  reliability  and  sample  size.   The  number  of  auditors  whose 
responses  indicated  the  highest  proportion  of  variance  was  attributed 
equally  to  IC-1  and  IC-3  were  3  and  6  for  the  reliability  and  sample 
size  judgments,  respectively.   Results  also  indicated  that  IC-3  was 
significant  (a £.05)  for  73  auditors  on  the  reliability  judgment  and 
for  62  auditors  on  the  sample  size  judgment.   Similarly,  IC-2  and  IC-1 
were  significant  for  42  and  34  auditors,  respectively,  on  the  relia- 
bility judgment  and  for  31  and  27  auditors,  respectively,  on  the  sample 
size  judgment. 
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The  fact  that  IC-3  accounted  for  most  of  the  variance  in  the 
auditors'  judgments  did  not  preclude  IC-1  and  IC-2  from  accounting  for 
much  of  the  variance  in  many  of  the  auditors'  judgments.   For  example, 
in  the  reliability  and  sample  size  judgments,  IC-3  accounted  for  the 
least  amount  of  variance  for  17  and  19  auditors,  respectively.   There- 
fore, there  is  considerable  disagreement  among  some  auditors  as  to  the 
relative  importance  of  the  IACs  evaluated  in  this  study.   This  point 
can  be  substantiated  with  respect  to  the  reliability  judgment  by 
examining  Table  4-5  where  omega-squared  values  for  six  of  the  auditors 
are  presented. 

The  auditors  were  selected  for  Table  4-5  as  examples  of  the 
variance  attributed  to  each  of  the  IACs  and  their  interaction  terms. 
The  results  are  shown  in  this  manner  to  indicate  the  relative  impor- 
tance of  IC-1,  IC-2,  and  IC-3  to  auditors  6-1,  109-4,  and  11-1, 
respectively,  and  the  relative  importance  of  certain  interaction  terms 
to  auditors  37-4,  101-4,  and  19-2.   As  indicated,  auditor  number  6-1 
relied  heavily  on  IC-1  and  to  some  extent  on  IC-3  but  to  no  signifi- 
cant extent  on  IC-2.   In  sharp  contrast,  auditors  109-4  and  11-1  relied 
almost  exclusively  on  IC-2  and  IC-3,  respectively,  when  making  the 
reliability  judgment. 

Referring  again  to  Table  4-3,  the  importance  of  interaction 
effects  can  be  seen.   There  were  15  auditors  who  had  at  least  one  sig- 
nificant (a  <  .10)  interaction  term  in  their  reliability  judgments. 
Three  examples  shown  in  Table  4-5  include  auditors  number  37-4,  101-4 
and  19-2  who  reflect  significant  interactions  for  IC-1 x IC-2,  IC-1  x 
IC-3,  and  IC-2  *IC-3,  respectively.   As  can  be  seen,  auditor  37-4 
relied  on  all  three  IACs  and  showed  two  significant  interactions, 
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TABLE  4-5 

SELECTED  EXAMPLES  OF  DIFFERENCES  IN  THE 
IMPORTANCE  OF  THE  IACs  TO  THE  VARIOUS  AUDITORS 

(RELIABILITY  JUDGMENT) 


Auditor 
Number 

Percentage  of  Total  Variance 
Accounted  For  by  IAC 

in  Jud 
s : 

gment 

IC-1 

IC-2 

IC-3 

IC-lxlC- 

2 

IC 

-lxIC- 

-3 

IC 

-2XIC-3 

6-1 

70.2 

a 

ns 

10.6 

ns 

10.6 

ns 

109-4 

3.4 

87.3 

3.4 

ns 

ns 

ns 

11-1 

ns 

ns 

93.3 

ns 

ns 

ns 

37-4 

17.4 

23.9 

31.3 

7.9 

ns 

11.9 

101-4 

44.4 

ns 

22.2 

ns 

22.2 

ns 

19-2 

ns 

21.1 

52.6 

ns 

ns 

21.1 

Not  significant  at  the  .10  level;  all  others  significant  at  a  5  .10. 
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while  the  results  for  auditor  101-4  indicate  no  significant  reliance 
on  IC-2  and  auditor  19-2  indicates  none  on  IC-1  for  the  reliability 
judgment.   However,  both  auditors  101-4  (22.2%)  and  19-2  (21.1%)  have 
significant  interaction  terms  which  account  for  a  large  percentage  of 
total  variance.   Table  4-5,  therefore,  provides  some  examples  of  the 
variation  in  responses  to  the  reliability  judgments. 

Table  4-6  provides  a  similar  analysis  for  the  sample  size 
judgment.   A  selection  of  auditors  provides  results  that  again  indicate 
the  differences  that  exist  with  respect  to  the  reliance  on  the  IACs 
when  making  the  sample  size  judgment.   IC-1  accounted  for  68.4%  of 
auditor  101-4's  variance,  IC-2  accounted  for  93.8%  of  auditor  15-2's 
variance,  and  88.9%  of  the  variance  in  auditor  41-4 's  sample  size 
judgment  came  from  IC-3.   For  all  of  these  auditors,  the  other  IACs 
and  interactions  were  not  significant. 

Interaction  effects  were  also  significant  for  many  auditors 
in  the  sample  size  judgment.   Nineteen  auditors  had  at  least  one  sig- 
nificant (a<.10)  interaction  term,  with  five  auditors  having  two 
significant  interaction  terms.   As  shown  in  Table  4-6,  auditors  88-4 
and  32-3  each  had  two  significant  interaction  terms  along  with  signif- 
icant main  effects  for  all  three  IACs.   Auditor  92-4  shows  a  signifi- 
cant interaction  for  IC-2* IC-3,  which  also  accounts  for  a  substantial 
percentage  of  his  total  variance  (29.1%).   Additional  sample  items 
from  Tables  4-3  and  4-4  could  provide  a  similar  basis  for  the  com- 
parisons discussed  above  and  summarized  in  Tables  4-5  and  4-6. 

The  relative  importance  of  the  IACs  and  their  two-factor  inter- 
actions are  summarized  in  Table  4-7.   The  importance  of  IC-3  within 
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TABLE  4-6 

SELECTED  EXAMPLES  OF  DIFFERENCES  IN  THE 
IMPORTANCE  OF  THE  IACs  TO  THE  VARIOUS  AUDITORS 

(SAMPLE  SIZE  JUDGMENT) 


Auditor  Percentage  of  Total  Variance  in  Judgment 

Number  Accounted  For  by  IACs: 

IC-1   IC-2   IC-3   IC-1  x ic-2    IC-1  x ic-3   IC-2  x ic-3 


101-4 

68.4 

ns 

ns 

15-2 

ns 

93.8 

ns 

41-4 

ns 

ns 

88.9 

88-4 

41.6 

12.3 

36.7 

32-3 

12.4 

8.1 

63.2 

92-4 

ns 

44.7 

20.1 

3.4 


ns 

ns 

ns 

ns 

5.1 

ns 

4.1 

8.1 

ns 

29.1 

Net  significant  at  the  .10  level;  all  others  significant  at  a  ^  .10. 
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the  specific  audit  setting  used  in  this  study  is  again  evident,  with 
evidence  for  IC-2  being  considered  more  important  than  IC-1.   The 
relative  importance  of  the  controls  is  maintained  whether  evaluating 
the  reliability  judgment  or  the  sample  size  judgment  and  also  for 
either  participants  with  significant  controls  and  interactions  or  for 
all  participants. 

The  fact  that  the  least  important  control  (IC-1)  accounts  for 
24.9%  of  the  variance  in  significant  sample  size  judgments  and  11.4% 
of  the  variance  in  all  sample  size  judgments  is  an  indication  that 
none  of  the  three  controls  was  disregarded  completely.   The  dependence, 
to  a  certain  extent,  on  all  three  controls  is  reflected  in  the  relative 
importance  of  the  interaction  terms.   For  significant  interactions,  the 
average  variance  was  14.9%  for  the  reliability  judgment  and  9.5%  for 
the  sample  size  judgment.   Comparisons  to  other  studies  should  be  made 
with  caution  due  to  the  specific  nature  of  the  audit  task  and  the 
inclusion  of  IACs  known  for  their  importance  in  making  the  required 
judgments.   However,  other  judgment  studies  in  auditing  have  shown 
average  variances  for  significant  interaction  terms  of  3.3%  [Joyce, 
1976J,  5%  and  5.1%  [Messier,  1979] ,  and  6.4%  [Ashton,  1974]. 

An  evaluation  of  the  variances  for  the  interaction  terms 
across  all  participants  indicates  average  variances  of  1-3%,  which  may 
be  considered  "normal"  for  this  type  of  task.   An  analysis  of  the 
information  pertaining  to  the  interaction  terms  indicates  that  few  of 
the  auditors  had  at  least  one  significant  (a  £.05)  interaction  term 
(5  of  85  for  reliability  judgment;  10  of  90  for  sample  size  judgment)4 

4 
The  weightings  of  the  interaction  terms  were  possible  for  80 
and  95  auditors,  respectively,  for  the  reliability  and  sample  size 
judgments  as  previously  indicated. 
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although  average  variances  were  high  for  the  significant  interactions. 
It  therefore  appears  that  some  auditors  processed  information  in  a 
highly  configural  manner,  although  the  number  was  few. 

The  various  firms  and  experience  levels  of  the  auditors  may  be 
suggested  as  factors  underlying  differences  in  responses.   Table  4-8 
provides  a  summary  of  the  omega-squared  values  by  firm  and  experience 
level.   Two  classification  schemes  are  used  in  presenting  information 
as  to  experience.   First,  a  comparison  is  made  between  those  auditors 
with  1-2  years  of  experience  and  those  with  3  or  more  years.   Secondly, 
those  with  3  years  of  experience  are  grouped  with  those  that  have  1-2 
years  and  contrasted  with  the  auditors  with  4  or  more  years  of  experi- 
ence.  The  values  that  are  shown  in  the  table  are  averages  that  were 
computed  by  summing  the  omega-squared  values  for  a  particular  firm  or 
experience  level  and  dividing  by  the  number  of  auditors  in  that  cate- 
gory.  The  total  of  102  auditors  for  the  reliability  judgment  and  101 
auditors  for  the  sample  size  judgment  for  which  the  ANOVA  analysis  was 
possible  are  reduced  to  100  and  99,  respectively,  for  the  experience 
classification  due  to  the  inability  to  properly  classify  two  of  the 
auditors. 

IC-3  appears  to  be  recognized  as  the  most  important  control 
for  the  reliability  decision  irrespective  of  the  firm  or  experience 
level.   However,  for  Firm  #2  the  difference  between  IC-3  and  IC-2  is 
negligible.   The  relative  importance  of  IC-3  is  maintained  for  the 
sample  size  judgment  across  all  classifications  except  for  Firm  #2, 
where  the  average  omega-squared  value  of  IC-2  (35.9)  was  greater  than 


See,  for  example,  auditors  88-4  and  89-4  in  Table  4-4. 
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TABLE  4-8 
VALUE  OF  STATISTICAL  WEIGHTS  BY  FIRM  AND  EXPERIENCE  LEVEL 

Average  Statistical  Weights  for 
Internal  Control  Variables: 

Firm  or                              Reliability 
Experience  No.  of  — • — 


Level    Subjects  IC-1   IC-2   IC-3   IC-1  x  IC-2   IC-1  x  IC-3   IC-2  x  ic-3 


Firm  #  1  (10)  11.5  16.3  56.6 

Firm   #  2  (10)  11.2  36.7  37.0 

Firm  #  3  (11)  8.1  26.3  48.0 

Firm  #4  (71)  13.1  20.6  46.2 

Exp.  E-Kl-2)  (49)  9.9  21.9  47.5 

Exp.  E-2(>3)  (51)  12.8  22.9  46.7 

Exp.  X-l  (1-3)  (79)  13.1  22.3  44.3 

Exp.  X-2(>4)  (21)  5.3  22.4  57.4 


Firm  #  1    (10)  11.2  14.9  55  3 

Firm  #  2    (10)  9.1  35.9  34.1 

Firm  //  3   (12)  11.0  27.9  37.2 

Firm  //  4    (69)  11.8  21.9  40.2 

Exp.  E-l  (1-2)  (50)  9.6  24.9  41.8 

Exp.  E-2  (>3)  (49)  12.2  21.5  40.2 

Exp.  X-l  (1-3)  (79)  11.8  24.9  38.1 

Exp.  X-2  (>4)  (20)  7.2  16.5  52  5 


1.5 

1.8 

2.0 

0.2 

0.9 

3.0 

0.9 

1.1 

1.7 

1.2 

1.3 

2.9 

1.2 

1.1 

3.5 

1.1 

1.6 

2.1 

1.0 

1.3 

3.2 

1.4 

1.5 

1.2 

Sample 

Size 

IC-1  x ic- 

2      IC- 

-1  x  ic-3 

IC-2  x ic-3 

2.2 

1.1 

3.7 

2.0 

1.2 

4.9 

0.1 

1.0 

3.5 

2.4 

1.7 

2.8 

1.1 

1.7 

3.6 

2.5 

1.3 

2.9 

1.9 

1.6 

3.1 

1.2 

1.5 

4.3 
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IC-3  (34.1).   This  was  due  primarily  to  the  omega-squared  values  of 
IC-2  (93.8)  and  IC-3  (0.5)  for  auditor  15-2.   Eliminating  his  values 
from  the  sample  size  judgments  of  Firm  //2  would  result  in  average 
values  of  29.5  (IC-2)  and  37.8  (IC-3)  for  the  remaining  nine  auditors. 

Of  additional  interest  with  respect  to  the  judgments  made  by 
the  auditors  is  the  total  variance  explained  by  the  individual  ANOVA 
models.   By  summing  the  total  omega-squared  values  for  the  IACs  and 
their  interaction  terms,  an  indication  of  the  overall  percentage  of 
variance  in  their  responses  that  is  accounted  for  by  the  IACs  and  inter- 
actions is  obtained.   Table  4-9  provides  a  distribution  of  auditors  by 
total  variance  explained. 

A  large  percentage  of  auditors  had  omega-squared  values  of  at 
least  80%.   Specifically,  79%  of  the  auditors  for  the  reliability 
judgment  and  61%  for  the  sample  size  judgment  had  total  omega-squared 
values  of  at  least  80%.   These  results  may  appear  misleading  when  com- 
pared to  the  omega-squared  values  computed  for  the  composite  judgments 
of  all  auditors  (45.4%  for  reliability  and  10.9%  for  sample  size). 
These  composite  results  indicate  some  disagreement  among  auditors  with 
respect  to  the  reliability  decision  and  substantial  disagreement  with 
respect  to  the  sample  size  decisions. 

The  findings  presented  in  this  section  with  regard  to  the 
description  of  the  judgments  made  in  this  study  generally  are  not 
surprising.   That  is,  there  was  some  disagreement  as  to  the  relative 
importance  of  the  IACs  in  terms  of  meeting  a  specific  objective,  and 
this  was  reflected  in  decisions  concerning  reliability  and  sample  size. 
On  the  other  hand,  there  was  general  agreement  that  IC-3  was  of  most 
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TABLE  4-9 
DISTRIBUTION  OF  SUBJECTS  BY  TOTAL  VARIANCE  EXPLAINED 


Range 

of 

Omega-Sc 

uar 

ed 

Rel 

lability 

Sample  Size 

Values 

Judgment 

Judgment 

.00-1 

.00 

48 

38 

.80- 

89 

33 

24 

.70- 

79 

10 

13 

.60  - 

69 

7 

14 

.50  - 

59 

2 

7 

< 

50 

2a 

5b 

ll  =  28.1         bl  =  35.9 

2=   4.9         2  =  35.8 

3  =  49.6 

4  =  36.0 

5  =  42.7 


Note:   Subjects  with  omega-squared  values  of  5.50 
had  little  variability  in  their  responses. 
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importance  in  making  the  required  decisions,  followed  by  IC-2  and  IC-1 
in  that  order.   As  anticipated,  most  of  the  variance  in  the  responses 
of  the  auditors  was  captured  by  a  linear  combination  of  the  IACs. 
The  interaction  terms  of  the  analysis  did  provide  some  of  the  total 
variance.   Indications  of  configural  processing  were  found  primarily 
in  individual  auditor  responses.   And  finally,  the  composite  model  of 
the  auditors'  judgments  did  a  very  poor  job  of  explaining  the  variation 
in  the  responses.   This  was  especially  true  of  the  sample  size  judg- 
ments. 

Judgment  Consensus 

The  extent  to  which  the  auditors  agreed  in  their  decisions 
given  the  same  set  of  data,  which  is  known  as  judgment  consensus,  was 
evaluated  through  the  use  of  canonical  correlation,  Pearson  product- 
moment  correlation,  and  cluster  analysis.   The  results  of  applying 
these  techniques  are  presented  in  this  section.   The  results  obtained 
from  performing  these  analyses  also  will  be  discussed  with  respect  to 
firm  and  experience  effects. 

The  ability  to  properly  evaluate  the  extent  of  consensus 
required  an  assumption  of  usable  and  reliable  responses  by  the 
auditors.   As  a  measure  of  the  auditors'  abilities  to  understand  the 
task  and  respond  in  an  internally  consistent  manner,  a  Pearson  product- 
moment  correlation  was  computed  across  the  12  cases  between  the  relia- 
bility and  sample  size  responses  for  each  auditor.   Given  the  nature 
of  the  decisions,  it  was  hypothesized  that  as  the  reliability  measure 
increased  (decreased),  the  suggested  sample  size  would  decrease 
(increase) .   A  negative  Pearson  product-moment  correlation  measure  was 


94 


assumed  to  indicate  "internal"  consistency.   Due  to  the  scale  of  the 
responses  for  reliability  (7-point  scale)  and  sample  size  (0  to  959), 
the  correlation  measures  varied  from  .37  to  -1.00,  with  an  average 
correlation  of  -.46.   The  responses  of  the  auditors  who  had  positive 
correlations  (there  were  a  total  of  6)  were  examined  for  reasonable- 
ness.  All  of  the  auditors  were  exhibiting  sufficient  internal  con- 
sistency to  consider  their  responses  as  conscientious  efforts,  except 
for  an  auditor  in  Firm  #1.   This  auditor's  responses  suggested  a  lack 
of  understanding  or  motivation  concerning  the  task  and  his  responses 
were  excluded  in  their  entirety  as  indicated  in  Chapter  III. 

Canonical  correlation.    The  extent  of  agreement  among  auditors 
with  respect  to  their  multivariate  decisions  on  reliability  and  sample 
size  was  evaluated  through  the  analysis  of  canonical  correlation  meas- 
ures.  Canonical  correlation  analysis  can  consider  the  reliability  and 
sample  size  responses  simultaneously,  enabling  the  derivation  of  a 
linear  combination  for  each  subject  such  that  the  correlation  between 
each  pair  of  subjects  is  maximized.   Since  109  auditors  participated 
in  the  study,  there  are  a  total  of  5,886  canonical  correlations  between 
all  pairs  of  subjects. 

The  mean  canonical  correlation  was  .78  with  a  range  of  .01  to 
1.00.   Similar  auditor  judgment  studies  that  have  computed  canonical 
correlations  show  similar  results.   For  example,  Joyce  [1976]  reported 
a  mean  correlation  of  .93  with  a  range  of  .36  to  1.00  and  Messier 
[1979]  reported  a  mean  correlation  of  .75  with  a  range  of  .24  to  .98. 
Although  these  correlations  all  appear  to  be  rather  high,  the  only 
conclusion  that  can  be  drawn  is  that  the  linear  combinations  of  one 
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particular  auditor's  judgments  seem  to  be  good  predictors  of  linear 
combinations  of  another  auditor's  judgments  in  these  specific  situa- 
tions.  An  assessment  of  the  variability  of  the  correlations  in  this 
study  resulted  in  a  standard  deviation  of  .145  which  compares  favorably 
to  the  standard  deviation  of  .17  reported  by  Messier  [1979].   However, 
it  still  provides  an  indication  of  some  variability  in  the  correlations. 

Pearson  product-moment  correlation.    Consensus  among  auditors 
also  can  be  evaluated  by  examining  the  Pearson  product-moment  correla- 
tion between  each  pair  of  auditors  for  both  the  reliability  and  sample 
size  judgments.   This  approach  resulted  in  two  measures  of  the  strength 
of  the  linear  relationship  of  the  responses  between  each  pair  of 
auditors.   The  mean  Pearson  product-moment  correlation  for  the  relia- 
bility judgment  was  .693  with  a  range  of  -.22  to  1.00.   The  sample 
size  judgment  resulted  in  a  mean  of  .609  with  a  range  of  -.66  to  1.00. 

The  Pearson  product-moment  correlations  found  in  this  study 
may  be  considered  rather  high  when  compared  to  Joyce  [1976],  who 
reported  an  average  consensus  measure  of  .373  with  a  range  of  -.68  to 
.93.   However,  there  have  been  similarly  high  consensus  measures 
obtained  in  audit  settings,  such  as  those  reported  by  Ashton  [1974] 
and  Messier  [1979].   Ashton  obtained  a  mean  correlation  of  .70  with  a 
range  of  .06  to  .93.   Messier,  in  measuring  both  a  materiality  and 
disclosure  judgment,  observed  mean  values  of  .665  and  .670  with  ranges 
of  -.10  to  .95  and  -.05  to  .98,  respectively. 

The  variability  of  the  correlation  measures,  as  reflected  in 
their  standard  deviations,  may  be  expected  to  be  high  due  to  the  wide 
ranges  of  results  indicated  above.   The  standard  deviation  of  the 
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correlations  for  the  reliability  and  sample  size  judgments  were  .19 
and  .22,  respectively.   The  finding  of  relatively  high  mean  correla- 
tions with  significant  variability  is  consistent  with  the  results  of 
the  canonical  correlation  analysis  discussed  previously. 

Firm  and  experience  effect.    To  evaluate  the  possible  impact 
of  firm  or  experience  differences  on  the  correlation  results,  average 
correlation  measures  were  calculated  by  firm  and  experience  level. 
Table  4-10  presents  the  average  canonical  correlation  by  firm  and 
experience  level.   As  indicated  previously,  two  possible  experience  level 
categorizations  are  utilized  to  determine  what  possible  effect  this 
may  have  on  the  analysis.   The  "E"  categories  contrast  experience 
levels  of  1-2  years  with  3  or  more  years,  while  "X"  categories  use  1-3 
years  versus  4  or  more.   The  values  found  in  Table  4-10  represent  the 
extent  of  agreement  among  auditors  within  the  same  firm  or  experience 
level  (diagonal  values)  or  the  extent  of  agreement  between  two  firms 
or  experience  levels  (off-diagonal  values).   Results  indicate  that  by 
combining  auditors  with  3  years  experience  with  the  group  with  1-2 
years  experience  as  opposed  to  those  with  4  or  more  years  experience, 
the  mean  canonical  correlation  of  responses  for  the  more  experienced 
auditors  increases  from  .766  to  .790. 

Table  4-11  provides  data  on  the  effect  of  firms  and  experience 
levels  on  the  reliability  and  sample  size  judgments  as  reflected  in 
Pearson  product-moment  correlation  measures.   These  results  are  con- 
sistent with  the  canonical  correlations.   That  is,  some  minor  varia- 
tions exist  between  firms  and  the  categorization  of  experience  levels 
can  have  an  effect  on  the  results.   All  firms  exhibited  relatively  high 
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TABLE  4-10 
JUDGMENT  CONSENSUS  AMONG  AUDITORS  BY  FIRM 


Firm 

(1) 

(2) 

(3) 

(4) 

(1) 

.770 

.785 

.797 

.776 

(2) 

.826 

.830 

.778 

(3) 

.827 

.781 

(4) 

.774 

JUDGMENT  CONSENSUS  AMONG  AUDITORS  BY  EXPERIENCE  LEVEL 


Experience  Level 

(E-l) 

(E-2) 

(E-l)    1-2  years 

.796 

.783 

(E-2)    3  or  more 

.766 

JUDGMENT  CONSENSUS  AMONG  AUDITORS  BY  EXPERIENCE  LEVEL 


Experience  Level 

(X-l) 

(X-2) 

(X-l)    1-3  years 

.785 

.783 

(X-2)    4  or  more 

.790 

Note:   The  values  in  this  table  are  mean  canonical  correlations. 
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TABLE  4-11 

JUDGMENT  CONSENSUS  AMONG  AUDITORS  BY  FIRM 
-  RELIABILITY  (SAMPLE  SIZE  JUDGMENT)  - 


Firm 

(1) 

(2) 

(3) 

(4) 

(1) 

.692(.612) 

.678(.587) 

.710(.629) 

•688(.603) 

(2) 

.72K.637) 

•739(.679) 

.688(.603) 

(3) 

.751(.708) 

.696(.612) 

(4) 

.688(.603) 

JUDGMENT  CONSENSUS  AMONG  AUDITORS  BY  EXPERIENCE  LEVEL 
-  RELIABILITY  JUDGMENT  - 


Experience  Level 

(E-1) 

(E-2) 

(E-1)   1-2  years 

.715 

.693 

(E-2)   3  or  more 

.685 

Experience  Level 

(X-1) 

(X-2) 

(X-1)   1-3  years 

.698 

.693 

(X-2)   4  or  more 

.734 

SAMPLE  SIZE  JUDGMENT 


Experience  Level 

(E-1) 

(E-2) 

(E-1)   1-2  years 

.651 

.609 

(E-2)   3  or  more 

.580 

Experience  Level 

(X-1) 

(X-2) 

(X-1)   1-3  years 

.619 

.609 

(X-2)   4  or  more 

.637 

Note:   The  values  in  this  table  are  mean  Pearson  correlations. 
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Pearson  product-moment  correlations  both  within  each  firm  (diagonal 
values)  and  between  each  combination  of  two  firms  (off-diagonal  values) . 
Firm  #3  had  the  highest  intra-firm  correlation  (r=.751)  and  had  inter- 
firm  correlations  with  the  other  firms  that  were  higher  than  the  intra- 
firm  correlations  of  any  other  firm. 

Of  additional  interest  in  Table  4-11  are  the  correlations  for 
the  various  experience  levels.   The  highest  correlations  are  obtained 
in  the  1-2  years  (E-l)  classification  and  the  4  or  more  (X-2)  classi- 
fication.  If  those  auditors  with  3  years  experience  are  grouped  with 
those  that  have  1-2  years  experience,  the  correlation  decreases  from 
.715  (.651)  to  .698  (.619)  for  the  reliability  (sample  size)  judgment. 
If  the  auditors  with  3  years  of  experience  are  grouped  with  those  that 
have  4  or  more  years  experience,  the  correlation  decreases  from  .734 
(.637)  to  .685  (.580)  for  the  reliability  (sample  size)  judgment. 

In  an  attempt  to  further  analyze  the  significance  of  firm  or 
experience  levels  on  the  judgments  made  by  the  auditors,  an  ANOVA 
analysis  was  performed.    The  ANOVA  analysis  combined  the  firm  factor 
with  the  three  IACs  for  the  evaluation  of  the  firm  significance  and 
the  experience  factor  (both  E  and  X  classifications)  with  the  three 
IACs  for  the  evaluation  of  the  significance  of  experience.   For  both 
the  reliability  and  sample  size  judgments,  the  firm  was  considered  as 
having  a  significant  (a £.01)  effect.   Using  the  "E"  classification 
scheme  (1-2  years  versus  3  or  more)  the  experience  level  effect  was 


A  combined  MANOVA  was  performed  that  indicated  the  firm 
affiliation  and  the  experience  level  were  significant  (a £.01).   MANOVA 
results  also  provided  evidence  that  the  three  IACs  were  significant 
(a  £.01),  as  were  two  of  the  interaction  terms:   IC-1  x  IC-3  (a<.05) 
and  IC-2xic-3  (ct<.01). 
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considered  significant  for  both  the  reliability  judgment  (a  <  .05)  and 
the  sample  size  judgment  (a  1.01).   In  contrast,  using  the  "X"  classi- 
fication scheme  (1-3  years  versus  4  or  more)  the  experience  level  was 
considered  significant  (a  1  .05)  only  for  the  reliability  judgment. 

An  additional  analysis  of  the  significance  of  differences  in 
responses  as  a  result  of  firm  and  experience  differences  is  shown  in 
Table  4-12.   These  results  are  similar  to  those  found  at  the  prelimi- 
nary audit  program  stage  and  presented  in  the  first  section  of  this 
chapter.   These  results  also  indicate  the  apparent  "conservatism" 
exhibited  by  Firm  #2  in  their  sample  size  decisions.   That  is,  their 
sample  size  selections  are  consistently  much  higher  than  those  of  the 
other  firms,  and  their  reliability  measures  are  comparable  if  not 
higher  in  some  cases.    Also,  the  responses  from  Firm  #4  are  comparable 
to  the  other  firms  with  respect  to  the  reliability  judgment,  yet  the 
auditors  indicate  on  the  average  much  smaller  sample  sizes.   The 
responses  of  Firm  #4  at  the  audit  program  revision  stage,  therefore, 
provide  an  interesting  contrast  to  those  of  Firm  #2. 

An  evaluation  of  the  average  responses  shown  in  Table  4-12 
according  to  the  various  experience  classifications  also  yields  some 
interesting  results.  An  examination  of  the  "E"  classifications  sug- 
gests that  the  more  experienced  auditors  were  less  conservative  in 
choosing  a  reliability  level  (e.g.,  consistently  greater  reliability 
across  all  levels  of  controls),  yet  consistently  chose  higher  sample 
sizes.   Results  from  examining  the  "X"  classifications  show  mixed 


It  is  important  to  remember  that  greater  reliability  is 
expected  to  result  in  smaller  sample  sizes,  not  larger  ones. 
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responses  when  comparing  the  more  experienced  auditors  to  the  less 
experienced. 

An  important  question  can  be  raised  with  respect  to  the 
responses  at  levels  2  and  3  of  IC-2.   That  is,  how  did  the  auditors 
perceive  the  distinction  between  a  specific  control  not  being  present 
(level  3)  versus  evidence  in  the  form  of  compliance  test  results  that 
indicate  a  control  is  not  operating  effectively  (e.g.,  results  show  a 
very  high  noncompliance  rate)  (level  2)?   As  could  probably  be  expec- 
ted, the  average  responses  for  levels  2  and  3  of  IC-2  show  mixed  reac- 
tions to  these  situations.   Overall,  results  indicate  auditors  feel 
the  reliabilities  of  the  IAC  systems  are  comparable,  although  slightly 
greater  sample  sizes  would  be  taken  if  the  control  were  not  present  as 
compared  to  the  control  not  operating  effectively.   In  contrasting 
individual  firms,  note  that  Firm  #4  provides  responses  for  the  relia- 
bility judgment  that  indicate  a  slightly  greater  reliability  for  level 
3  as  compared  to  level  2  for  IC-2,  whereas  the  other  firms'  responses 
indicate  that  more  of  a  difference  does  exist.   However,  the  responses 
of  Firm  #2  and  Firm  #3  suggest  a  perception  that  level  2  represents  a 
more  reliable  system  than  level  3;   the  auditors  in  Firm  //l  indicated 
just  the  opposite.   The  sample  size  responses  for  Firm  #1  (116.31; 
115.42)  and  Firm  #4  (92.29;  93.69)  are  very  similar  for  the  two  levels. 
However,  both  Firm  #2  (207.63;  224.08)  and  Firm  #3  (114.52;  175.23) 
show  average  responses  that  indicate  a  much  larger  sample  size  is 
needed  in  the  situation  where  IC-2  is  not  present  than  when  it  is 
considered  not  to  be  operating  effectively. 
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Cluster  analysis.   The  cluster  analysis  was  performed  to 
determine  to  what  extent  the  auditors  who  were  grouped  together  in 
the  analysis  had  any  similar  characteristics.   The  data  used  for 

Q 

clustering  the  auditors  into  groups  were  the  standardized   responses 
to  the  reliability  and  sample  size  decisions  for  the  twelve  cases. 
Cluster  analysis  was  run  on  the  reliability  and  the  sample  size 
responses  individually,  and  also  on  the  judgments  taken  together. 

The  results  of  the  cluster  analysis  provide  for  another 
measure  of  the  degree  of  consensus.   The  results  indicate  that  the 
groupings  for  the  reliability  judgment  bear  no  relationship  to  those 
for  the  sample  size  judgment.   This  result  is  not  surprising  given 
the  large  scaling  difference  between  the  two  responses.   Furthermore, 
the  standardization  of  the  responses  could  not  keep  the  sample  size 
decisions  from  dominating  in  the  combined  cluster  analysis.   In  other 
words,  the  clusters  that  resulted  from  using  the  standardized  relia- 
bility and  sample  size  decisions  together  were  the  same  as  those  that 
resulted  from  using  the  sample  size  decisions  only. 

A  summary  of  the  cluster  analysis  is  presented  in  Table  4-13. 
The  results  for  the  movement  from  four  clusters  to  two  clusters  indi- 
cate that  there  are  some  outliers  (18,  20,  27,  81)  for  the  sample 
size  and  combined  analysis.  A  classification  of  the  auditors  in  each 
cluster  as  to  firm  affiliation  and  experience  level  could  not  explain 
the  make-up  of  the  clusters.  No  specific  conclusion  can  be  stated 
concerning  consensus  from  the  results  of  the  cluster  analysis. 


Standardization  was  accomplished  by  taking  the  mean  response 
for  each  auditor  across  the  twelve  cases  (for  both  the  reliability  and 
sample  size  judgments)  and  subtracting  each  of  his  responses. 
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TABLE  4-13 
SUMMARY  OF  CLUSTER  ANALYSIS 


-  SAMPLE  SIZE  AND  RELIABILITY  RESPONSES  - 

Cluster  Auditors 

No.  of  clusters  =4    1.1,  2,  7,  11,  15,  16,  17,  26,  29,  35,  46,  48, 

49,  55,  61,  63,  74,  85,  95 

2.  All  others 

3.  18,  20,  27 

4.  81 

No.  of  clusters  =2    1.   All  others 

2.   18,  20,  27,  81 


-  RELIABILITY  RESPONSES  - 

Cluster  Auditors 

No.  of  clusters  =4    1.   All  others 

2.  1,  3,  8 

3.  5,  18,  20,  26,  35,  55,  61,  65,  67,  68,  76, 
82,  97,  102,  103 

4.  32,  36,  41,  46,  52,  64,  75,  78,  87,  105,  107 

No.  of  clusters  =2    1.   All  others 

2.   32,  36,  41,  46,  52,  64,  75,  78,  87,  105,  107 
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Self-Insight 

Self-insight  indices  were  computed  for  each  auditor  to  examine 
the  extent  of  agreement  between  the  auditor's  subjective  evaluation  of 
the  relative  importance  of  the  three  key  IACs  in  making  his  reliability/ 

sample  size  decisions  and  their  importance  as  reflected  in  the  statis- 

q 
tical  weights  determined  through  the  ANOVA  results.    The  auditors' 

subjective  weights  were  elicited  directly  in  the  post-experiment  ques- 
tionnaire by  asking  each  auditor  to  allocate  100  points  among  the 
three  IACs  according  to  their  relative  importance  to  the  reliability/ 
sample  size  decisions  in  the  12  cases  just  completed.  The  statistical 
weights  used  in  arriving  at  the  self-insight  indices  were  the  adjusted 
omega-squared  values  for  the  three  key  IACs;  these  reflect  the  propor- 
tion of  total  variance  in  the  auditor's  judgment  accounted  for  by  each 
control . 

The  use  of  the  omega-squared  values  for  both  the  reliability 
and  sample  size  judgments  resulted  in  two  sets  of  statistical  weights 
for  each  auditor  and,  therefore,  enabled  self-insight  indices  for  the 
reliability  judgment  and  for  the  sample  size  judgment  to  be  obtained. 
These  indices  are  discussed  in  greater  detail  in  the  following  sec- 
tions.  First,  the  subjective  weights  given  by  the  auditors  are 
discussed.   Then,  the  results  of  the  computation  of  the  self-insight 
indices  are  presented  and  discussed. 


9 
Self-insight  indices  could  be  calculated  for  only  those 
auditors  who  responded  to  the  appropriate  questions  for  subjective 
weights  and  those  for  which  omega-squared  values  were  available  from 
the  ANOVA  analysis.   Three  subjects  failed  to  indicate  subjective 
weights  in  their  post-experiment  questionnaire. 
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Subjective  weights.    The  weights  assigned  to  each  IAC  by  the 
auditors  are  presented  in  Table  4-14.   A  comparison  of  the  overall 
average  responses  for  each  factor  indicates  that  IC-3,  with  an  average 
weight  of  48.1,  is  considered  the  most  important.   IC-2  (28.9)  ranked 
a  distant  second,  although  thought  to  be  slightly  more  important  than 
IC-1  (23.0).   A  comparison  of  these  averages  with  those  shown  in  Table 
4-7  indicates  that  as  a  group  the  auditors  were  able  to  assess  the 
relative  importance  of  their  most  important  cue  very  successfully, 
although  perhaps  slightly  overstating  its  importance.   The  importance 
of  IC-2  to  the  auditors'  judgments  also  was  reasonably  well  recognized 
by  the  auditors  themselves.   However,  consistent  with  similar  previous 
research  dealing  with  human  judgment  [Slovic  and  Lichtenstein,  1971], 
the  auditors  tended  to  overestimate  the  relative  importance  of  the 
least  important  cues.   In  this  study,  though  IC-1  was  considered  by 
the  auditors  to  be  least  important,  an  average  subjective  weight  of 
23.0  was  provided  by  them;  on  the  other  hand,  the  ANOVA  results  indi- 
cated average  statistical  weights  of  12.2  and  11.4  for  the  reliability 
and  sample  size  judgments,  respectively. 

A  summary  of  the  subjective  weights  by  firm  and  experience 
level  is  provided  in  Table  4-15.   A  comparison  of  firms  indicates  that 
members  of  Firm  #1  and  Firm  //4  weighted  the  importance  of  IC-3  slightly 
higher  than  did  the  other  firms,  and  correspondingly  weighted  IC-2 
slightly  lower.   The  lowest  rating  of  IC-1  was  provided  by  Firm  #1; 
Firm  #2  weighted  IC-1  lower  than  Firm  #4  due  to  the  greater  subjective 
weight  given  to  IC-2  by  Firm  #2.   Average  subjective  weights  for  all 
firms  did  show  that  IC-3  was  considered  the  most  important  followed  by 
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TABLE  4-14 
VALUE  OF    SUBJECTIVE   WEIGHTS   BY   AUDITOR 


Subj 

active 

Weight 

Subj 

active 

Weight 

for 

IC  Variable 

for 

IC  Variable 

Subj 

-  Firm 

IC-1 

IC-2 

IC-3 

Sub.i 

-  Firm  | 

IC-1 

IC-2 

IC-3 

1 

-   1 

20 

10 

70 

31 

-  3 

50 

25 

25 

2 

-   1 

15 

35 

50 

32 

-  3 

20 

10 

70 

3 

-   1 

5 

70 

25 

33 

-  3 

20 

40 

40 

4 

-   1 

5 

45 

50 

34 

-  3 

5 

25 

70 

5 

-   1 

35 

U  5 

20 

35 

-  4 

20 

35 

45 

G 

-   1 

60 

0 

40 

36 

-  4 

30 

20 

50 

7 

-   1 

10 

30 

60 

37 

-  4 

25 

50 

25 

8 

-   1 

30 

35 

35 

38 

-  4 

35 

40 

25 

9 

-   1 

20 

30 

50 

39 

-  4 

50 

20 

30 

10 

-   1 

10 

15 

75 

40 

-  4 

20 

35 

45 

11 

-   1 

0 

0 

100 

41 

-  4 

10 

20 

70 

12 

-   1 

20 

30 

50 

42 

-  4 

15 

30 

55 

13 

-   2 

15 

40 

45 

43 

-  4 

35 

25 

40 

14 

-   2 

15 

35 

50 

44 

-  4 

35 

15 

50 

15 

-   2 

30 

20 

50 

45 

-  4 

20 

30 

50 

16 

-   2 

20 

35 

45 

46 

-  4 

15 

15 

70 

17 

-   2 

25 

30 

45 

47 

-  4 

10 

10 

80 

18 

-   2 

20 

70 

10 

48 

-  4 

20 

40 

40 

19 

-  2 

20 

30 

50 

49 

-  4 

20 

30 

50 

20 

-  2 

35 

30 

35 

50 

-  4 

40 

20 

40 

21 

-  2 

30 

20 

50 

51 

-  4 

40 

20 

40 

22 

-  2 

10 

25 

65 

52 

-  4 

30 

20 

50 

23 

-  3 

20 

35 

45 

53 

-  4 

30 

30 

40 

24 

-  3 

10 

40 

50 

54 

-  4 

25 

25 

50 

25 

-  3 

30 

20 

50 

55 

-  4 

60 

20 

20 

26 

-  3 

45 

35 

20 

56 

-  4 

20 

30 

50 

27 

-  3 

20 

40 

40 

57 

-  4 

20 

20 

60 

28 

-  3 

15 

35 

50 

58 

-  4 

20 

30 

50 

29 

-  3 

40 

20 

40 

59 

-  4 

10 

50 

40 

30 

-  3 

25 

50 

25 

60 

-  4 

10 

40 

50 
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TABLE  4-14 

-  continued 

Subjective 
for  IC  Vai 

We  ight 
iable 

Subjt 
for 

active 
IC  Var 

Weight 
iable 

Subj  - 

Firm 

I  IC-1 

IC-2 

IC-3 

Subj 

-  F 

irm  | 

IC-1 

IC-2 

IC-3 

61   - 

4 

30 

20 

50 

85 

- 

4 

60 

15 

25 

62   - 

4 

10 

50 

40 

86 

- 

4 

5 

0 

95 

63  - 

4 

5 

5 

90 

87 

- 

4 

10 

30 

60 

64  - 

4 

5 

40 

55 

88 

- 

4 

75 

20 

5 

65  - 

4 

25 

50 

25 

89 

- 

4 

30 

20 

50 

66  - 

4 

20 

40 

40 

90 

- 

4 

20 

40 

40 

67  - 

4 

35 

40 

25 

91 

- 

4 

40 

10 

50 

68  - 

4 

5 

65 

30 

92 

- 

4 

30 

20 

50 

69  - 

4 

10 

40 

50 

93 

- 

4 

30 

10 

60 

70  - 

4 

10 

20 

70 

94 

- 

4 

20 

30 

50 

71   - 

4a 

95 

- 

4 

10 

15 

75 

72  - 

4 

30 

20 

50 

96 

- 

4 

10 

20 

70 

73  - 

4 

15 

70 

15 

97 

- 

4 

25 

35 

40 

74   - 

4 

10 

30 

60 

98 

- 

4 

10 

20 

70 

75   - 

4 

0 

35 

65 

99 

- 

4 

30 

40 

30 

76  - 

4 

33 

33 

34 

100 

- 

4a 

77  - 

4 

20 

20 

60 

101 

- 

4 

50 

15 

35 

78  - 

4 

0 

10 

90 

102 

- 

4a 

79  - 

4 

50 

20 

30 

103 

- 

4 

40 

40 

20 

80  - 

4 

10 

33 

55 

104 

- 

4 

20 

40 

40 

81   - 

4 

20 

50 

30 

105 

- 

4 

0 

0 

100 

82  - 

4 

10 

40 

50 

106 

- 

4 

30 

20 

50 

83  - 

4 

20 

40 

40 

107 

- 

4 

0 

5 

95 

84  - 

4 

20 

35 

45 

108 

- 

4 

35 

15 

50 

109 

- 

4 

50 

25 

25 

Overall 

23.0 

28.9 

48.1 

These   subjects  did   not   respond    to    this    question    in   the   post-experiment 
quest ionaire. 
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TABLE  4-15 
VALUE  OF  SUBJECTIVE  WEIGHTS  BY  FIRM  AND  EXPERIENCE  LEVEL 


Firm  or 
Experience     Average  Subjective  Weights  for  Internal  Controls 

Level  IC-1  ic-2  IC-3 


Exp, 


Firm 

Firm  #1  19.2  28.8  52.1 

Firm  #2  22.0  33.5  44.5 

Firm  #3  25.0  31.3  43.8 

Firm  #4  23.4  28.0  48.6 


E-l  (1-2  years)  24.2  27.6  48.2 

E-2  (3  or  more)  21.9  30.2  47.8 

X-l  (1-3  years)  23.7  28.7  47.6 

X-2  (4  or  more)  20.5  29.8  49.5 

Overall  23.0  28.9  48.1 
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IC-2  and  IC-1,  in  that  order.   These  results  are  similar  to  those  for 
the  statistical  weights  that  are  presented  in  Table  4-8. 

The  average  subjective  weights  by  experience  level  indicate 
general  agreement  as  to  the  relative  importance  of  IC-3  over  the  other 
controls.   Although  IC-2  and  IC-1  are  considered  second  and  third  in 
order  of  importance  for  all  experience  levels,  more  experienced 
auditors  seemed  to  attach  greater  importance  to  IC-2  and  less  impor- 
tance to  IC-1  than  did  the  lesser  experienced  auditors.   There  is  no 
ready  explanation  as  to  why  auditors  with  more  experience  would  either 
attach  greater  significance  to  shipment  authorization  (IC-2)  and/or 
less  significance  to  credit  approval  (IC-1)  for  this  audit  situation. 

Self-insight  index.    In  order  to  more  thoroughly  examine  the 
relationship  between  the  subjective  weights  discussed  in  the  previous 
section  and  the  statistical  weights  determined  from  the  ANOVA  analysis, 
a  self-insight  index  was  computed  for  each  auditor.   Since  each 
auditor  made  both  reliability  and  sample  size  judgments,  two  self- 
insight  indices  were  computed  for  each  auditor  for  which  ANOVA  was 
possible. 

The  self-insight  indices  resulted  from  calculating  the  Pear- 
son product-moment  correlation  measures  between  the  subjective  and 
statistical  weights  across  the  three  key  IACs.   As  discussed  previously, 
the  statistical  weights  used  were  "adjusted"  weights  that  included 
interaction  effects.     Caution  is  recommended  in  evaluating  these 

See  p.  55  above  for  an  explanation  of  the  method  used  to 
arrive  at  the  "adjusted"  statistical  weights. 
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results  since  these  correlations  were  computed  over  only  three  vari- 
ables and  may  therefore  be  considered  unstable. 

Table  4-16  shows  the  self-insight  indices  for  each  auditor. 
These  indices  vary  considerably  from  a  low  of  -1.0  to  a  high  of  +1.0. 
The  self-insight  index  was  much  higher  for  the  reliability  judgment 
than  for  the  sample  size  judgment.   For  example,  51.5%  of  the  auditors 
had  a  reliability  index  greater  than  .90.   This  percentage  increased 
to  63.6%  for  those  with  an  index  greater  than  .80.   For  the  sample 
size  judgment,  only  38.8%  of  the  auditors  had  an  index  greater  than 
.90,  with  the  percentage  increasing  to  52.0%  for  those  with  an  index 
greater  than  .80. 

The  overall  mean  self- insight  indices  were  .68  and  .59  for 
the  reliability  and  sample  size  judgments,  respectively.   These  results 
are  not  as  high  as  most  studies  involving  auditor  judgment.   For 
example,  Ashton  [1974]  reported  a  mean  Pearson  correlation  of  .89,  and 
extensions  of  his  study  obtained  mean  correlations  of  .77  [Ashton  and 
Kramer,  1980]  and  .86  [Ashton  and  Brown,  1980].   Hamilton  and  Wright 
[1977]  and  Messier  [1979]  showed  results  which  also  indicated  a  high 
degree  of  self-insight.   The  results  in  the  present  study  are  most 
similar  to  those  of  Joyce  [1976]  who  reported  a  mean  self-insight  of 
.53  with  a  range  of  -.78  to  1.00. 

The  self-insight  indices  also  were  summarized  by  firm  and 
experience  levels  as  shown  in  Table  4-17.   The  participants  from  Firm 
#1  showed  substantially  greater  self-insight  than  those  from  other 
firms.   A  possible  explanation  lies  in  the  fact  that  nine  of  the 
eleven  auditors  from  Firm  #1  for  which  self-insight  indices  were 
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TABLE  4-16 
SELF-INSIGHT  INDICES  FOR  INDIVIDUAL  AUDITORS 


Self -Insight 

Index 

Self-Insight 

Index 

Auditor  - 

Sample 

Auditor  - 

Sample 

Firm 

Reliability 

Size 

Firm 

Reliability 

Size 

1  -  1 

a 

.99 

31-3 

a 

.99 

2  -  1 

.88 

.85 

32-3 

.99 

.99 

3  -  1 

.63 

.56 

33-3 

.98 

.75 

4-1 

.98 

.62 

34-3 

.98 

.91 

5-1 

.98 

.99 

35-4 

.12 

-.31 

6-1 

.90 

.74 

36-4 

.97 

.99 

7  -  1 

.95 

.99 

37-4 

•  45 

.91 

8  -  1 

.77 

.47 

38-4 

.76 

.76 

9-1 

.98 

.96 

39-4 

.76 

.76 

10-1 

a 

a 

40-4 

.89 

.82 

11-1 

1.00 

a 

41-4 

.99 

.99 

12-1 

.97 

.96 

42-4 

.98 

.97 

13-2 

.95 

.52 

43-4 

.69 

-.32 

14-2 

.86 

.99 

44-4 

.94 

.52 

15-2 

-.76 

-.76 

45-4 

.98 

.31 

16-2 

.74 

.99 

46-4 

.99 

.99 

17  -  2 

.97 

.99 

47-4 

.99 

a 

18-2 

.99 

.99 

48-4 

.87 

.62 

19-2 

.96 

.98 

49-4 

.95 

.96 

20-2 

.27 

.18 

50-4 

-.19 

.02 

21-2 

.91 

.76 

51-4 

.39 

.67 

22-2 

.82 

.86 

52-4 

.98 

.96 

23-3 

.79 

.81 

53-4 

.95 

.84 

24  -  3 

.94 

.99 

54-4 

.76 

-.50 

25-3 

-.35 

.27 

55-4 

.65 

-.68 

26-3 

.97 

.11 

56-4 

.99 

.92 

27-3 

.99 

.85 

57-4 

-.12 

.98 

28-3 

.95 

.98 

58-4 

-.97 

.53 

29-3 

.71 

-.05 

59-4 

.99 

.50 

30-3 

.07 

-.22 

60-4 

.87 

.86 

ANOVA  results  did  not  allow  for  determination  of  statistical  weights. 


TABLE  4-16  -  continued 
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Self-Insight 

Index 

Self-Insight 

Index 

Auditor  - 

Sample 

Auditor  - 

Sample 

Firm 

Reliability 

Size 

Firm 

Reliability 

Size 

61-4 

-.31 

-.70 

86-4 

a 

a 

62-4 

.85 

.79 

87-4 

.94 

.93 

63-4 

a 

a 

88-4 

.31 

.52 

64-4 

.78 

.77 

89-4 

.85 

.38 

65-4 

-.31 

-.100 

90-4 

.96 

.70 

66-4 

.65 

.68 

91-4 

.80 

.93 

67-4 

.79 

.77 

92-4 

.94 

-.14 

68-4 

.99 

.99 

93-4 

.87 

.27 

69-4 

.95 

.87 

94-4 

.93 

.97 

70-4 

a 

a 

95-4 

.99 

.99 

71-4 

b 

b 

96-4 

.40 

.99 

72-4 

.96 

.94 

97-4 

-.68 

.87 

73-4 

1.00 

-1.00 

98-4 

-.10 

-.92 

74-4 

.99 

.97 

99-4 

-.52 

.89 

75-4 

.86 

.89 

100  -  4 

b 

b 

76-4 

.50 

.50 

101  -  4 

.99 

.71 

77-4 

.99 

.71 

102  -  4 

b 

b 

78-4 

.99 

a 

103  -  4 

.75 

.64 

79-4 

.06 

.14 

104  -  4 

.47 

.60 

80-4 

.94 

.99 

105  -  4 

a 

a 

81-4 

.99 

.99 

106  -  4 

.94 

.95 

82-4 

-.98 

-.99 

107  -  4 

.99 

.86 

83-4 

.75 

.99 

108  -  4 

.99 

.98 

84-4 

.99 

.81 

109  -  4 

-.50 

-.16 

85-4 

.99 

.94 

Overall 

.68 

.59 

ANOVA  results  did  not  allow  for  determination  of  statistical  weights. 


Auditor  did  not  provide  the  necessary  subjective  weights. 
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TABLE  4-17 
SELF- INSIGHT  INDICES  BY  FIRM  AND  EXPERIENCE  LEVEL 


Firm  or 

Experience 

Level 


Firm  # 1 

Firm  //  2 

Firm  #  3 

Firm  #  4 


Average  Value  of  Self-Insight  Index 
Reliability  Sample  Size 


.90 
.67 
.73 

.64 


.81 
.65 
.62 

.55 


Exp. 


E-l  (1-2  years) 

E-2  (3  or  more) 

X-l  (1-3  years) 

X-2  (4  or  more) 


.62 
.74 

.65 

78 


.55 
.64 
.56 
.73 


Overall 


,59 
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calculated  had  4  or  more  years  of  experience  and  all  were  participants 
in  an  advanced  statistical  sampling  seminar  at  the  time  of  the  current 
study.   When  all  firms  are  considered  and  either  experience  classifi- 
cation scheme  is  used,  the  more  experienced  auditors  had  higher  self- 
insight  indices  for  both  the  reliability  and  sample  size  judgments. 
For  the  reliability  judgment,  the  more  experienced  auditors  (E-2  and 
X-2)  had  indices  of  .74  and  .78  as  compared  to  .62  and  .65  for  the 
lesser  experienced  auditors  (E-l  and  X-l) .   Similar  results  are  shown 
for  the  sample  size  judgment  where  indices  of  .64  and  .73  were  obtained 
for  the  more  experienced  auditors  as  compared  to  .55  and  .56  for  the 
lesser  experienced  auditors.   Results  from  the  studies  conducted  by 
Ashton  [1974],  Hamilton  and  Wright  [1977],  Ashton  and  Brown  [1980], 
and  Ashton  and  Kramer  [1980]  indicate  that  judgment  insight  increases 
with  increased  levels  of  auditing  experience.   Greater  familiarity 
with  the  task  provides  a  possible  explanation  for  the  superior  insight 
of  those  auditors  with  at  least  3  years  of  experience. 

Additional  Data 

Additional  data  is  available  to  better  understand  the  judgments 
of  the  auditors  participating  in  the  current  study.   First,  results 
are  provided  to  indicate  the  approach  taken  by  the  auditors  in  using 
the  statistical  sampling  tables  in  arriving  at  the  sample  size  deci- 
sion.  Summary  comments  are  then  presented  to  reflect  the  responses 
to  questions  in  the  post-experiment  questionnaire. 

Decision  making  approach.    The  selection  of  a  sample  size 
from  the  statistical  tables  required  the  auditors  to  choose  an 
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appropriate  materiality  and  confidence  level  for  each  case.   The 
materiality  and  confidence  levels  chosen  by  each  auditor  across  the 
12  cases  were  examined  and  are  summarized  in  Table  4-18.   Again,  the 
familiarity  of  statistical  sampling  techniques  by  the  participants  in 
Firm  //l  may  underlie  their  decision  to  maintain  a  constant  materiality 
across  all  cases.   This  approach  can  be  considered  the  most  appropri- 
ate in  the  application  of  a  statistical  sampling  plan. 

The  approach  taken  by  most  of  the  auditors  from  Firm  #3  and 
Firm  #4  can  be  contrasted  to  that  approach  taken  by  members  of  Firm 
#1.   Most  of  these  auditors  chose  to  vary  both  the  materiality  and 
confidence  level  across  the  12  cases.   No  strong  firm  preference  was 
revealed  by  members  of  Firm  #2.   A  possible  explanation  for  these 
results  can  be  found  by  examining  some  results  of  the  answers  to  the 
post-experiment  questionnaire. 

Questionnaire  results.   The  post-experiment  questionnaire 
provided  the  opportunity  for  the  auditors  to  respond  in  some  impor- 
tant areas.   Three  specific  areas  of  interest  were  the  representive- 
ness  of  the  task,  the  auditors'  familiarity  with  the  task,  and  the 
extent  to  which  similar  decisions  are  made  by  the  auditor.   Overall, 
the  task  was  considered  "moderately  representative,"  receiving  an 
average  rating  of  3.0  on  a  5-point  scale.   The  auditors'  familiarity 
with  the  task  and  the  extent  to  which  they  make  similar  decisions  also 
were  rated  by  the  auditors.   With  a  rating  of  3  representing  "moder- 
ately familiar"  and  a  4  representing  "not  very  familiar,"  the  average 
rating  of  task  familiarity  was  3.25.   A  contrast  between  Firm  #1 
(2.50)  and  Firm  //3  (3.67)  again  is  very  much  in  evidence.   Finally, 
with  a  3  representing  "sometimes"  and  a  4  representing  "often"  with 
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TABLE  4-18 
SAMPLE  SIZE  DECISIONS  -  BY  FIRM 


Firm 
Decision  Basis         Firm  ill        Firm  f/2    Firm  #3   Firm  #4   Total 


Constant 
Materiality  (M) 


Constant 

Confidence  Level  (CL) 


Variation  in 
M  and  CL 


18       30 


1         3         2        11       17 


44       60 


Total       12        10        12        75      109 


Two  subjects  responded  with  decisions  of  either  0  or  a  number  that 
reflects  both  a  constant  materiality  and  confidence  level. 
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respect  to  the  extent  of  similar  decisions,  the  overall  average  was  3.04. 
The  experience  level  and  use  of  statistical  sampling  by  the  auditors  in 
Firm  #1  is  evidenced  by  their  average  rating  of  similar  decisions  of 
3.67,  as  compared  to  that  of  Firm  #3  of  3.17.   These  results  provide 
some  explanation  as  to  the  differences  between  responses  of  Firm  #1  and 
Firm  #3  members  that  were  referred  to  in  the  previous  section. 

Eight  post-experiment  questions  were  included  to  reflect  the 
extent  of  motivation  of  the  auditors  in  performing  the  experimental 
task.   The  analysis  of  responses  to  these  questions  reflects  a  substan- 
tial difference  between  those  auditors  in  the  lowest  quartile  (low 
motivation)  and  those  in  the  highest  quartile  (high  motivation).   A 
substantial  difference  is  also  found  between  the  mean  self-insight 
indices  in  the  highest  quartile  (.87)  and  those  in  the  lowest  quartile 
(.48). 

Summary 

The  results  of  the  study  were  presented  and  briefly  discussed 
in  this  chapter.   An  examination  of  the  responses  from  the  preliminary 
audit  program  stage  revealed  some  differences  among  both  the  individual 
auditors  and  the  participating  firms.   Also,  the  evaluation  of  the 
auditors'  responses  to  the  compliance  test  decisions  suggested  that 
IC-3  was  considered  the  most  important  IAC,  with  IC-2  and  IC-1  follow- 
ing in  that  order. 

Responses  provided  at  the  audit  program  revision  stage  provided 
the  opportunity  for  the  primary  and  more  extensive  analysis.  Specific- 
ally, ANOVA  results  were  used  to  evaluate  the  relative  importance  of 
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the  key  internal  controls  provided  in  this  specific  reliability /sample 
size  judgment.   In  addition,  the  extent  of  consensus  among  the  auditors 
was  examined  with  results  summarized  by  firm  and  experience  level. 
Self-insight  indices  also  were  calculated  and  summarized  by  firm  and 
experience  level.   Finally,  other  information  pertaining  to  the  decision 
processes  used  by  the  auditors  was  presented,  followed  by  some  results 
of  a  post-experiment  questionnaire. 

Results  of  analyzing  the  responses  at  the  preliminary  audit 
program  stage  are  summarized  as  follows: 

1.  Differences  among  the  firms  were  noted  with  respect 
to  the  reliability  and  sample  size  responses. 

2.  Responses  to  the  compliance  test  decisions  suggested 
that  IC-3  was  considered  the  most  important  control, 
followed  by  IC-2  and  IC-1,  in  that  order. 

Analysis  of  responses  to  the  twelve  case  situations  presented 
at  the  audit  program  revision  stage  produced  results  as  follows: 

1.  IC-3  (control  over  physical  shipment)  accounted  for  most 
of  the  variance  in  the  auditors'  judgments,  with  IC-2 
(shipment  authorization)  and  IC-1  (credit  approval) 
following  in  order  of  importance. 

2.  Individual  auditor  responses  indicated  that  approximately 
62%  of  the  auditors  had  most  of  the  variance  accounted  for 
by  IC-3,  approximately  24%  of  the  auditors  had  IC-2  account 
for  most  of  the  variance  in  their  responses,  and  the 
remaining  14%  of  the  auditors  had  IC-1  account  for  most 

of  the  variance. 
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3.  Significant  interaction  effects  were  found  for  fifteen 

of  the  102  auditors  on  the  reliability  and/or  sample  size 
decisions, 

4.  Evaluation  of  judgment  consensus  resulted  in  a  mean  canon- 
ical correlation  of  .78  with  a  standard  deviation  of 
responses  of  .145.   These  results  represent  reasonable 
agreement  among  the  auditors,  but  nevertheless  reveal 
some  variability  among  their  responses. 

5.  Another  measure  of  consensus  was  obtained  from  the  use  of 
Pearson  product-moment  correlation.   The  mean  correlation 
for  the  reliability  judgment  was  ,693  with  a  standard 
deviation  of  .19,   The  sample  size  judgment  had  a  mean 
correlation  of  .609  with  a  standard  deviation  of  .22. 

6.  The  degree  of  self-insight  exhibited  by  the  auditors  can 
be  considered  somewhat  low  when  compared  to  other  account- 
ing studies.   The  mean  values  for  the  reliability  and 
sample  size  judgments  were  .68  and  .59,  respectively. 

7.  Firm  and  experience  levels  were  considered  in  examining 
the  above  results.   In  most  cases,  firm  and/or  experience 
levels  were  shown  to  impact  on  the  results. 

The  following  chapter  will  discuss  these  results  in  greater 
detail  in  conjunction  with  some  implications,  suggestions  for  future 
research,  and  conclusions. 


CHAPTER  V 
SUMMARY,  CONCLUSIONS,  AND  SUGGESTED  FUTURE  RESEARCH 

Summary  of  the  Problem  and  Research  Approach 

This  dissertation  has  provided  additional  evidence  to  facili- 
tate a  better  understanding  of  the  role  of  auditor  judgment  in  the 
evaluation  of  IAC  information  and  the  decision  as  to  the  extent  of 
subsequent  audit  procedures  (substantive  tests).   A  laboratory  experi- 
ment was  conducted  to  gather  data  which  were  used  to  examine  auditor 
judgment  about  the  reliability  of  an  IAC  system  and  the  resulting 
selection  of  a  sample  size  for  a  substantive  test.   The  participants 
were  109  members  of  four  "Big-Eight"  accounting  firms.   A  descriptive 
approach  was  taken  within  the  framework  provided  by  the  Brunswik  lens 
model  to  examine  these  decisions  for  a  specific  audit  task. 

Although  other  studies  have  examined  similar  decisions  made  by 
auditors,  some  specific  and  significant  limitations  were  thought  to 
exist  in  these  studies.   The  present  study  sought  to  overcome  those 
limitations.   Primarily,  a  more  realistic  setting  was  used,  with  a 
greater  degree  of  control  over  the  significant  factors  impacting  upon 
the  decisions  of  the  auditors.   These  improvements  were  discussed  in 
detail  in  Chapter  II  of  this  dissertation. 

Two  key  decision  points  within  the  audit  process  that  require 
the  exercise  of  professional  judgment  were  examined  in  this  study. 
These  included:   (1)  the  preliminary  audit  planning  stage  and  (2)  the 
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audit  program  revision  stage.   Examination  of  the  responses  from  these 
stages  resulted  in  descriptions  of  the  auditors'  judgments  and  the 
evaluation  of  auditor  consensus  and  self-insight.   However,  the  extent 
of  auditor  consensus  is  the  focal  point  in  this  study  for  examining 
the  exercise  of  professional  judgment  within  the  audit  process. 

There  are  a  number  of  reasons  to  expect  and/or  hope  for  a 
certain  level  of  consensus  among  auditors  who  are  applying  their  profes- 
sional judgment  to  the  same  audit  task.   First,  most  auditors  have 
similar  educational  backgrounds  in  accounting.   In  addition,  accounting 
firms  provide  additional  training  that  lends  itself  to  a  common  basis 
for  decision  making  among  the  auditors.   Finally,  to  become  a  Certified 
Public  Accountant  an  individual  is  required  to  pass  a  uniform  exam  by 
providing  responses  that  indicate  adherence  to  a  prescribed  set  of 
standards  established  by  the  accounting  profession.   Therefore,  it 
appears  that  the  lack  of  consensus  among  auditors  is  considered  more 
costly  than  the  efforts  to  achieve  an  accepted  level  of  agreement. 

Lack  of  consensus  among  auditors  can  be  considered  costly  for 
a  number  of  reasons.   A  lack  of  consensus  could  suggest  that  the  deci- 
sions of  some  auditors  are  either  resulting  in  unnecessary  and  costly 
audit  work  or  that  the  firm  is  subjecting  itself  to  additional  risk 
(cost)  as  a  result  of  insufficient  audit  work.   Additionally,  the 
auditors  who  are  making  less  than  "optimal"  decisions  are  sharing 
their  "expertise"  with  others  in  the  firm  since  a  large  degree  of 
learning  in  auditing  is  done  "on  the  job."  Finally,  lack  of  consensus 
could  lead  to  excessive  and  costly  review  procedures  that  would  be 
required  in  addition  to  those  that  are  normally  present.   The  descrip- 
tion of  the  auditors'  judgments  and  the  computation  of  self-insight 
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indices  are  related  directly  to  the  topic  of  consensus.   Results  from 
this  study  in  all  of  these  areas  are  presented  in  the  following  section. 

Summary  of  the  Results  of  the  Study 

A  word  of  caution  should  be  presented  prior  to  discussing  the 
results  of  the  current  study.   Weber  [1978,  p.  384]  notes  that  any 
findings  derived  from  a  fixed  effects  model  can  not  be  generalized. 
For  this  reason,  and  others  that  were  enumerated  earlier,  generaliza- 
tion of  these  results  beyond  the  participating  auditors  and  the 
specific  task  should  be  avoided.   However,  since  a  realistic  setting 
was  provided  as  representative  of  the  type  of  reliability  and  sample 
size  decisions  that  are  commonly  made  by  auditors,  meaningful  state- 
ments can  still  be  made  from  evaluating  the  results  of  the  study. 

Description  of  Auditors'  Judgments 

The  description  of  the  auditors'  decisions  both  at  the  prelimi- 
nary audit  program  stage  and  at  the  audit  program  revision  stage  indi- 
cated that  the  most  important  control  in  deciding  that  recorded  sales 
were  for  valid  transactions  was  one  concerning  control  over  physical 
shipment  (IC-3) ,  followed  by  the  controls  for  shipment  authorization 
(IC-2)  and  credit  approval  (IC-1) ,  respectively.   The  fact  that  the 
least  important  control  (IC-1)  accounted  for  24.9%  of  the  variance  in 
significant  (a-  .05)  sample  size  judgments  and  11.4%  of  the  variance 
in  all  sample  size  judgments  made  at  the  program  revision  stage  is  an 
indication  that  each  of  the  internal  controls  was  considered  important. 
IC-3  was  recognized  as  the  most  important  control  irrespective  of  the 
firm  or  experience  level  chosen  for  the  reliability  and  sample  size 
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decisions,  except  for  the  sample  size  decisions  of  Firm  #2  where  IC-2 
was  considered  slightly  more  important.   The  importance  of  these  con- 
trols, as  discussed  above,  must  be  considered  the  "relative"  importance 
within  the  context  of  the  current  study. 

The  Extent  of  Judgment  Consensus 

The  extent  of  consensus  among  auditors  at  the  audit  program 
revision  stage  was  evaluated  through  the  use  of  canonical  correlation, 
Pearson  product-moment  correlation,  and  cluster  analysis.   The  mean 
canonical  correlation  was  high  with  a  value  of  .78,  but  also  showed 
high  variability  with  a  standard  deviation  of  .145.   The  mean  Pearson 
product-moment  correlations  for  the  reliability  and  sample  size  judg- 
ments were  .69  and  .61,  respectively.   The  standard  deviations  for 
these  judgments  were  again  quite  high  at  .19  and  .22,  respectively. 
These  results  indicate  a  limited  amount  of  consensus  among  the  partic- 
ipating auditors.   Analysis  of  firm  and  experience  levels  showed  these 
factors  to  have  a  significant  impact  on  the  extent  of  consensus  among 
auditors.   The  cluster  analysis  results  were  inconclusive  as  to  the 
extent  of  consensus  among  the  auditors  or  as  to  the  impact  of  firm  or 
experience  differences. 

In  commenting  on  the  level  of  consensus  found  in  this  study 
there  are  two  very  important  issues  that  should  be  addressed.   First, 
the  reasons  for  any  lack  of  consensus  must  be  examined.   Second,  the 
costs  associated  with  the  various  levels  of  consensus  must  be  known 
and  properly  evaluated. 

While  the  level  of  consensus  obtained  in  this  study  appears 
low,  the  relative  complexity  of  the  judgment  task  must  be  considered  a 
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contributing  factor.   The  absence  in  this  study  of  any  review  process 
or  joint  decision  making  that  would  be  found  in  actual  practice  is 
another  possible  explanation.   Other  possible  reasons  can  be  found  in 
the  results  that  indicated  there  were  differences  among  firms  and 
between  experience  levels.   That  is,  different  auditing  philosophies 
or  the  training  provided  by  the  individual  firms  was  evident.   Also, 
the  extent  of  an  individual's  practical  experience  impacted  on  the 
extent  of  consensus. 

The  current  descriptive  study  was  not  concerned  with  examining 
the  costs  of  various  levels  of  consensus  as  a  basis  for  establishing 
this  as  a  major  problem  within  the  auditing  profession.   Any  implica- 
tions from  the  results  presented  above  with  respect  to  the  extent  of 
consensus  among  the  participating  auditors  are  difficult  to  discern. 
As  discussed  in  the  previous  section,  there  are  many  indications  that 
the  auditing  profession  desires  a  certain  level  of  consensus  among  its 
members.   However,  the  results  of  this  study  present  no  indication  of  a 
need  for  "corrective  action"  from  individual  accounting  firms  or  the 
accounting  profession  to  increase  the  current  level  of  consensus. 

The  Degree  of  Self-Insight 

A  comparison  of  the  auditor's  subjective  evaluation  of  the 
relative  importance  of  the  three  key  IACs  and  the  importance  as 
reflected  in  the  statistical  weights  was  obtained  through  a  self- 
insight  index.   The  overall  mean  self-insight  indices  were  .68  and 
.59  for  the  reliability  and  sample  size  judgments,  respectively. 
Greater  self-insight  was  reflected  by  those  with  more  years  of 
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experience.   The  ability  to  understand  and  express  a  decision  approach 
is  most  important  for  the  auditors  participating  in  this  study.   As 
seniors  with  in-charge  responsibilities,  the  auditors  have  responsi- 
bility for  communication  skills  both  to  their  subordinates  and  to 
their  superiors.   An  extensive  amount  of  learning  takes  place  "on  the 
job"  and  provides  the  auditor  with  a  basis  for  formulating  an  audit 
decision  approach.   For  these  reasons,  the  low  self-insight  indices 
in  the  current  study  are  a  source  of  some  concern.   The  fact  that  the 
auditors  were  responding  to  a  relatively  complex  judgment  task  is  a 
possible  explanation  for  the  somewhat  low  indices  calculated  in  this 
study. 

Implications  for  the  Auditing  Profession 

In  comparing  this  study  with  similar  auditing  studies,   results 
substantiate  previous  findings  in  some  areas  and  provide  some  new  and 
meaningful  findings  in  other  areas.   As  an  important  extension  of 
earlier  studies,  this  study  has  examined  a  greater  portion  of  the 
audit  revision  process  and  in  greater  detail  than  has  been  done 
before.   A  significant  finding  in  this  study  was  some  evidence  that 
differences  existing  among  auditors  and/or  firms  at  the  preliminary 
audit  program  stage  appear  to  continue  to  exist  at  the  audit  program 
revision  stage.   Of  course,  this  finding  requires  additional  research 
support  prior  to  suggesting  any  immediate  implications. 


These  studies  would  include  Ashton  [1974],  Joyce  [1976], 
Mock  and  Turner  [1979]  and  others  referred  to  in  Chapter  II  above. 
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Although  previous  studies  have  used  various  audit  tasks,  the 
level  of  consensus  found  in  this  study  was  comparable  to  that  obtained 
in  other  audit  studies.   The  use  of  statistical  sampling  techniques 
and  a  rather  complex  audit  task  enhanced  the  contribution  of  this  study. 
The  auditing  profession  should  be  most  interested  in  the  impact  of 
introducing  additional  realism  into  these  types  of  studies  and  evalu- 
ating the  results.   In  addition,  information  as  to  the  "perceived" 
importance  of  key  internal  controls  in  examining  various  other  audit 
objectives  can  be  obtained  through  the  study  of  similar  audit  tasks. 

The  various  approaches  taken  by  auditors  to  determine  a  statis- 
tical sample  can  provide  meaningful  information  to  auditors,  firms,  and 
to  the  auditing  profession.   Findings  in  this  study  indicate  that 
differences  do  exist  among  auditors  in  their  application  of  statistical 
sampling  techniques.   To  the  extent  that  statistical  sampling  is  a 
significant  portion  of  the  decision  making  aspect  of  the  audit  process, 
the  ability  of  auditors  to  use  this  tool  should  be  appropriately 
monitored. 

In  summary,  perhaps  the  primary  implication  from  this  study 
is  that  it  is  not  obvious  that  the  level  of  consensus  desired  by  the 
auditing  profession  is  being  achieved.   Of  course,  there  were  indica- 
tions that  more  consensus  was  being  achieved  within  the  firms  and  at 
more  experienced  levels,  which  may  be  considered  more  important  than 
across  the  entire  auditing  profession.   A  major  concern  is  whether  or 
not  resources  are  being  properly  utilized  to  enhance  the  probability 
of  consensus  among  auditors.   For  this  reason,  the  auditing  profession 
must  be  aware  of  any  evidence  that  suggests  the  current  level  of 
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consensus  among  auditors  is  not  optimal  with  respect  to  the  utilization 
of  available  resources. 

Additionally,  perhaps  both  the  extent  of  consensus  and  self- 
insight  could  be  enhanced  if  more  attention  was  given  to  providing 
feedback  to  the  auditors  with  respect  to  their  "on  the  job"  decisions. 
As  mentioned  in  the  previous  section  a  first  step  would  be  to  determine 
the  reasons  for  such  differences,  and  then  to  perform  a  cost-benefit 
analysis  to  allow  for  the  comparison  of  alternatives.   With  "on  the 
job"  feedback  both  difficult  and  costly,  additional  consideration 
should  be  given  to  the  approach  currently  taken  in  firm  training  ses- 
sions to  provide  meaningful  feedback. 

The  current  study  is  one  of  many  recent  studies  that  must  be 
built  upon  and  extended  to  provide  the  necessary  impetus  for  system- 
atically examining  the  audit  process.   In  recent  years  members  of  the 
auditing  profession  and  academic  researchers  have  come  to  realize  that 
the  complexity  of  the  audit  process  does  not  preclude  it  from  being 
subjected  to  appropriate  research  examination.   Such  suggestions  for 
future  research  relating  to  the  current  study  are  provided  in  the 
following  section. 

Suggestions  for  Future  Research 

Since  this  study  was  not  intended  to  be  an  end  in  itself,  a 
number  of  possible  avenues  for  future  research  exist  in  this  area. 
Many  opportunities  exist  to  examine  other  portions  of  the  audit  proc- 
ess that  would  involve  use  of  different  internal  accounting  controls 
and/or  audit  objectives.   Other  adjustments  in  a  similar  study  would 
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include  the  manner  in  which  information  is  provided  to  the  auditors  or 
the  types  of  decisions  required.  Elicitation  of  subjective  probabili- 
ties seems  to  be  a  viable  alternative  approach  to  examine  the  decision 
making  process  of  auditors.  Also,  the  fact  that  auditors  are  involved 
in  the  sequential  processing  of  information  contributes  to  the  attrac- 
tiveness of  a  Bayesian  approach. 

Audit  decisions  are  also  made  at  various  levels  within  the 
accounting  firm.   Use  of  managers  or  partners  in  similar  audit  situa- 
tions might  provide  greater  insight  into  the  extent  of  consensus  on 
"final"  decisions  versus  those  made  "up  front"  by  those  auditors  with 
considerably  less  experience.   Similar  research  could  be  conducted 
using  audit  teams  as  subjects  which  incorporates  a  more  realistic 
group  decision  making  situation  and  would  incorporate  the  possibility 
of  some  type  of  peer  review.   Incorporation  of  such  a  realistic  environ- 
ment, however,  can  lead  to  many  additional  research  problems  not  faced 
when  using  individual  responses. 

A  broader  approach  that  could  be  used  to  examine  this  problem 
area  would  include  an  attempt  to  determine  what  factors  seem  to 
enhance  consensus  among  auditors.  This  information,  used  in  conjunc- 
tion with  information  about  costs  associated  with  attaining  consensus 
versus  the  costs  for  unacceptable  lov?  levels  of  consensus,  could 
affect  some  standard  operating  procedures  within  the  auditing  profes- 
sion. 

A  final  area  that  warrants  additional  auditor  judgment  research 
deals  with  the  development  of  an  operational  model  that  might  better 
establish  the  relationship  between  internal  accounting  control 
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evaluation  and  the  resulting  extent  of  substantive  testing.   The  testing 
of  such  a  model  to  determine  its  usefulness  in  audit  decision  making 
situations  would  also  be  warranted.   This  type  of  research  would  lend 
itself  to  what  has  been  reflected  as  a  major  concern  and  premise  of 
this  study,  that  is,  inefficient  allocation  of  resources  within  an 
audit . 

The  role  of  professional  judgment  has  been  and  will  continue 
to  be  a  major  part  of  the  audit  process.   However,  the  need  to  continue 
to  examine  this  process  and  to  expose  it  to  systematic  and  meaningful 
research  will  not  diminish  in  the  near  future.   This  research  has  been 
presented  with  the  hope  that  it  can  in  some  way  serve  as  a  stepping 
stone  for  future  research  in  this  vitally  important  area. 
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APPENDIX 
THE    EXPERIMENTAL   MATERIALS 


UNIVERSITY  OF  FLORIDA 


GAINESVILLE  32611 
904-392-0155 


Dear  Participant: 

I  would  like  to  take  this  opportunity  to  thank  you 
for  participating  in  this  study.   A  number  of  other 
auditors  will  also  be  participating  in  the  study.   In 
reporting  the  results  of  this  study  all  individuals  and 
their  associated  firms  will  remain  anonymous. 

Your  careful  consideration  of  the  information  provided 
and  the  questions  asked  will  help  to  ensure  reliable  and 
meaningful  results. 

Again,  thank  you  for  your  time  and  assistance  in  this 
pro j  ec  t . 


S  incerely , 

Rick    Tabor 
Ph.D.     Candidate 
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INTRODUCTION 


The  purpose  of  this  study  is  to  examine  auditor  judgment  in  a 
routine  audit  situation.   Specifically,  I  am  interested  in  your 
consideration  of  the  key  factors  of  internal  control  evaluation  and 
compliance  test  results  in  determining  the  extent  of  substantive  tests 
to  be  performed. 

Please  read  carefully  the  following  information  and  attempt 
to  make  your  decisions  in  terms  of  the  setting  provided.   I  realize 
that  the  following  materials  do  not  include  all  the  information  you 
might  like  to  have  for  your  decisions.   However,  as  with  many  real 
situations,  it  is  not  always  possible  to  have  all  the  information  you 
would  like.   For  this  reason,  please  respond  to  the  best  of  your 
ability  on  the  basis  of  the  information  that  has  been  provided. 
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INSTRUCTIONS 

Assume  that  you  have  just  been  assigned  as  senior-in-charge  of  the 
audit  of  Tabkol  Manufacturing  Company  (TMC) .   This  is  the  first  year 
that  TMC  has  required  an  audit  and  your  firm  was  chosen  to  provide  this 
audit  for  the  year  ended  December  31,  1978.   Unfortunately  for  you,  the 
senior-in-charge  who  bad  started  the  preliminary  work  for  this  job  in 
September  is  no  longer  with  your  firm.   It  is  now  time  for  the  prelimi- 
nary work  to  be  completed  and  the  year-end  work  to  be  planned,  and  you 
have  been  asked  to  take  over  at  this  point. 

This  study  is  concerned  with  planning  the  amount  of  compliance  and 
substantive  testing  that  is  appropriate  within  the  sales  cycle  at  year- 
end  after  evaluating  the  information  available  from  the  preliminary  work 
completed  in  September.  Most  importantly,  you  should  be  concerned  with 
only  one  objective  while  planning  your  audit  program  relative  to  sales. 
That  is,  the  only  audit  objective  within  the  context  of  this  study  will 
be  that,  "recorded  sales  are  for  valid  transactions." 

Most  of  the  information  you  will  be  presented  is  unchanging  back- 
ground information  about  TMC  and  other  unchanging  information  gathered 
during  the  preliminary  audit  work.   Some  of  the  information  about  TMC 
(specifically,  three  key  internal  controls  whose  existence  gives  assur- 
ance concerning  the  audit  objective  and  compliance  test  results 
concerned  with  these  controls)  is  not  held  constant.   These  factors 
(key  internal  controls  and  compliance  test  results)  are  varied  to  form 
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12  different  hypothetical  situations  you  might  encounter  in  your  audit 
of  the  sales  cycle  of  TMC.   You  will  be  asked  to  provide  the  appropriate 
sample  sizes  for  compliance  and  substantive  tests  in  the  preliminary 
audit  program  and  a  final  sample  size  for  a  substantive  test  after  con- 
sidering the  specific  information  presented  in  each  case  situation  along 
with  the  unchanging  background  information. 

The  Procedures  To  Follow  In  This  Study  Are  As  Follows:* 

(1)  Review  the  information  provided  to  this  point  (Instructions  and 
Introduction)  and  seek  any  clarifications. 

(2)  Carefully  familiarize  yourself  with  the  Background  Information 
and  Flowchart  that  follows,  and  is  common  to  all  cases,  before 
proceeding. 

(3)  Examine  the  additional  information  presented  for  each  scenario 
and  answer  the  questions  for  the  scenario  and  the  appropriate 
cases  before  proceeding  to  the  next  scenario  and  doing  the  same. 


*You  may  review  the  Introduction,  Instructions,  Background  Information, 
and/or  Flowchart  at  any  time;  however,  these  need  not  be  reviewed  for 
each  different  scenario  or  case  situation. 
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BACKGROUND  INFORMATION 

-  Tabkol  Manufacturing  Company  (TMC)  is  a  medium-sized  firm  which 
manufacturs  hand  tools  such  as  hammers  and  screwdrivers.   Its  products 
are  distributed  nationally  through  commission-paid  representatives  who 
sell  building  materials,  power  tools,  and  other  items  to  hardware 
stores  and  building  supply  outlets.   The  firm's  products  are  generally 
displayed  on  racks  by  the  retailers. 

-  TMC  has  experienced  an  annual  growth  rate  in  sales  of  approximately 
10  percent  over  the  past  several  years.   The  client  currently  has 
approximately  6,000  regular  customers.   These  customers  average  about 
eight  orders  per  year  (therefore,  approximately  48,000  orders  are  proc- 
essed yearly),  and  the  average  order  size  is  about  $200.   Thus,  sales 
(all  are  credit  sales)  amounted  to  almost  $10  million  last  year. 
Accounts  receivable  comprise  nearly  34  percent  of  the  total  assets  and 
there  is  no  inventory  on  consignment. 

-  The  previous  auditor  anticipated  that  the  balance  sheet  at  December 
31,  1978,  would  closely  resemble  the  following: 

Cash                 $   500,000  Accounts  payable  $   700,000 

Accounts  receivable    1,400,000  Accrued  taxes  100,000 

Inventory             1,200,000  Bank  debt  1,500,000 

Plant  and  equipment    1,000,000  Equity  1,800,000 

$4,100,000  $4,100,000 


144 


-  Furthermore,  the  previous  auditor  anticipated  that  profit  would 
amount  to  about  $600,000  before  taxes  on  the  $10  million  in  sales. 

-  The  accompanying  flowchart*  reflects  the  operations  of  the  sales 
cycle  for  TMC.   Note  the  representation  of  the  three  key  internal 
controls  (IC-1,  IC-2,  and  IC-3)  within  the  chart.   It  is  important  to 
remember  that  the  existence  of  these  controls  may  vary  among  the  dif- 
ferent cases. 

-  Sales  returns  and  allowances  for  TMC  are  too  immaterial  to  include 
in  the  accompanying  flowchart  or  to  verify  in  the  audit. 


*The  flowchart  was  constructed  as  a  result  of  discussions  with  key 
personnel. 
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NARRATIVE  TO  ACCOMPANY  FLOWCHART 

Telephone  orders  are  received  from  customers  by  the  sales  order 
department.   A  clerk  prepares  a  customer  order  document  (checking 
prices,  computing  the  dollar  amount  of  the  order,  etc.).   Another  clerk 
checks  the  approved  customer  list  provided  by  the  Accounting  Department 
for  credit  approval.   The  customer  order  is  initialed  to  indicate 
approval,  or  if  the  customer  is  not  on  the  list,  the  order  is  sent 
directly  to  the  Accounting  Department  for  approval  or  notice  to  the 
customer  that  prepayment  is  required. 

An  invoice  is  prepared  in  two  copies  and  the  customer  order  accom- 
panies the  shipping  copy  of  the  invoice.   The  customer  copy  of  the 
invoice  (Copy  1)  is  sent  to  the  billing  department  and  held  in  the 
"pending"  file  awaiting  notification  that  the  order  was  shipped.   Noti- 
fication is  indicated  by  the  receipt  of  the  shipping  copy  of  the 
invoice  with  the  supporting  customer  order  and  bill  of  lading. 

The  billing  clerk  matches  the  received  shipping  copy  with  the 
customer  copy  from  the  pending  file.   Both  copies  of  the  invoice  are 
priced,  extended,  and  footed.   The  customer  copy  is  then  mailed  directly 
to  the  customer  and  the  shipping  copy  (with  the  supporting  customer 
order  and  bill  of  lading)  is  sent  to  the  accounts  receivable  clerk. 
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Specific  Audit  Objective:   Recorded  sales  are  for  valid  transactions. 


Description  of  Key  Internal  Controls:   (These  may  or  may  not  be 

present  in  the  following 
case  materials.) 


IC-1:   Procedures  exist  to  control  for  credit  approval  as  indicated  by 
proper  initials  on  supporting  customer  order.   A  customer  order 
is  initialed  to  indicate  credit  approval  if  the  customer  is  on 
the  approved  credit  list  provided  by  the  Accounting  Department 
or  is  approved  through  the  alternative  procedures.   (See  the 
narrative  accompanying  the  flowchart.) 

-  Noncompliance  implies  lack  of  proper  initials  on  the  customer 
order. 


Expected  Noncompliance  Rate  for  IC-1  =  2% 


IC-2:   Procedures  exist  to  control  for  shipment  authorization  as  indi- 
cated by  proper  initials  on  the  sales  invoice.   A  sales  invoice 
is  initialed  to  indicate  that  the  sales  invoice  is  in  agreement 
(i.e.,  as  to  customer  name,  quantity  of  goods,  price,  etc.)  with 
the  customer  order  and  that  the  shipping  department  is  authorized 
to  make  the  appropriate  shipment. 

-  Noncompliance  implies  lack  of  proper  initials  on  the  sales 
invoice . 


*  Expected  Noncompliance  Rate  for  IC-2  =  2% 


IC-3:   Procedures  exist  to  control  for  the  physical  shipment  of  goods 
as  indicated  by  proper  initials  on  the  bill  of  lading.   Copy  4 
of  the  bill  of  lading  is  initialed  to  indicate  the  goods  were 
actually  shipped  and  were  supported  by  a  bill  of  lading  (Copy  2) 
that  agreed  with  the  shipment  as  to  the  name  of  the  customer  and 
quantity  of  goods. 

-  Noncompliance  implies  lack  of  proper  initials  on  Copy  4  of 
the  bill  of  lading. 


*  Expected  Noncompliance  Rate  for  IC-3  =  2% 


*  These  expected  noncompliance  rates  are  based  upon  a  review  of  the 
Sales  Cycle  Flowchart  and  a  preliminary  assessment  of  the  controls 
believed  to  be  operating. 


NOTE:   Other  important  controls  relative  to  this  objective  include: 
1)  the  review  and  follow-up  of  customer  complaints  resulting 
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from  the  receipt  of  a  monthly  statement  and  2)  the  prenumbering 
of  documents.   These  controls  will  not  be  formally  tested  as  it 
was  determined  through  alternative  procedures  (i.e.,  observation 
of  and  discussions  with  key  personnel  and  a  general  review  of 
the  documents)  that  the  few  complaints  that  are  received  are 
very  infrequent  and  not  material  and  that  all  documents  are  pre- 
numbered  and  that  numerical  integrity  appears  to  be  maintained. 

Also,  you  may  assume  that  normal  review  procedures  will  be 
followed  in  such  related  areas  as  inventory  cut-off,  review  of 
aged  receivables,  uncollectibles,  etc. 
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-   SCENARIO  #1    - 
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SCENARIO  #1  -  The  following  information  is  common  to  CASES  la  through 
lh.   Please  disregard  any  information  concerning 
SCENARIO  #2. 

The  internal  control  questionnaire  shows  the  following  results  relative 
to  the  existence  of  the  key  controls  discussed  in  the  information  pro- 
vided previously. 


IC-1 :  YES,  procedures  exist  to  control  for  credit  approval. 

IC-2:  YES,  procedures  exist  to  control  for  shipment  authorization. 

IC-3:  YES,  procedures  exist  to  control  for  the  physical  shipment  of  goods. 


QUESTION:   As  a  result  of  all  the  information  you  have  been  provided 
concerning  TMC  to  this  point  (including  the  specific  infor- 
mation in  this  scenario  concerning  the  existence  of  the 
three  key  controls),  what  degree  of  reliability  would  you 
assign  to  the  given  controls  with  regard  to  meeting  the 
specific  objective  of  interest?  (Circle  the  appropriate 
number. ) 

Moderately      Moderately       Extremely 
No      Low     Low  High    High    High 

Reliability  Rel.     Rel .  Rel.     Rel.     Rel. 


The  following  statements  require  your  response  as  to  the  most  appropri- 
ate sample  sizes  in  a  preliminary  audit  program  (See  NOTE  below.)  for 
both  compliance  and  substantive  procedures.   Using  the  information 
provided  in  the  accompanying  booklet  regarding  various  sample  sizes  for 
compliance  and  substantive  tests,  complete  the  following  statements. 
(You  must  select  a  sample  size  from  those  provided.) 

Compliance  Tests  -  In  the  selection  of  sample  sizes  for  the  compliance 
tests,  please  treat  each  test  individually  although  the  sampling  unit 
(sales  invoice  packet)  will  be  the  same  for  each  test  and  as  a  practical 
matter  could  be  combined. 

CT-1 ,  CT-2,  &  CT-3:     Sample  size  selection  and  evaluation  tables 

(Tables  1  &  2)  are  statistically  determined. 

CT-1 :   (Test  of  IC-1):   I  would  recommend  that  sales  invoices 

be  examined  to  insure  that  the  credit  approval 
procedures  have  been  followed  (indicated  by 
the  appropriate  initials  on  the  supporting 
customer  order. 

-  This  decision  was  based  upon  a  desired  confidence  level  of  %, 

(From  Table  1) 
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CT-2 :   (Test  of  IC-2):   I  would  recommend  that  sales  Invoices 

be  examined  to  insure  that  the  shipment  autho- 
rization procedures  have  been  followed  (indi- 
cated by  the  appropriate  initials  on  the  sales 
invoice) . 

-  This  decision  was  based  upon  a  desired  confidence  level  of  _  %. 

(From  Table  1) 

CT-3:   (Test  of  IC-3) :   I  would  recommend  that  sales  invoices 

be  examined  to  insure  that  proper  control  pro- 
cedures over  the  physical  shipment  of  goods 
have  been  followed  (indicated  by  proper  initials 
on  the  supporting  bill  of  lading). 

-  This  decision  was  based  upon  a  desired  confidence  level  of  %. 

(From  Table  1) 

Substantive  Test   (Sample  size  Table  3  is  statistically  derived  using  a 
variables  sampling  approach.) 

As  a  result  of  the  information  I  have  been  provided  and  in  conjunction 
with  my  planned  compliance  tests,  I  would  recommend  as  part  of  a  preli- 
minary audit  program  the  selection  of  customers  from  the  customer 

accounts  file  as  of  11/30/78  and  that  positive  confirmations  be  sent  to 
those  selected.   (You  must  select  a  sample  size  from  Table  3.) 


NOTE:   In  the  context  of  this  study,  "preliminary  audit  program"  implies 
an  initial  audit  program  that  will  possibly  be  adjusted  for  the 
substantive  test  as  a  result  of  the  evaluation  of  compliance  test 
results.   At  that  time,  a  "final  audit  program"  is  determined. 
Also,  the  substantive  test  (confirmation  of  receivables)  will  be 
performed  on  an  interim  basis  as  of  11/30/78  with  compliance 
testing  performed  for  the  period  1/1/78-11/30/78.   Appropriate 
additional  procedures  will  be  performed  for  the  month  of  Decem- 
ber. 
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CASE  la 

-  Assume  that  your  recommended  sample  sizes  have  been  used  in  the 
compliance  testing  of  the  internal  controls  that  were  found  to  be 
present . 

The  results  of  the  compliance  tests  can  be  summarized  as  follows: 

CT-1 :  Noncompliance  rate  =  8% 
CT-2 :  Noncompliance  rate  =  2% 
CT-3:   Noncompliance  rate  =  2% 


QUESTION:   As  a  result  of  all  the  information  you  have  been  provided 

concerning  TMC  to  this  point  (including  the  specific  results 
of  your  suggested  compliance  tests  provided  above),  what 
degree  of  reliability  would  you  assign  to  the  given  controls 
with  regard  to  meeting  the  specific  objective  of  interest? 
(Circle  the  appropriate  number. ) 


Moderately 
No      Low     Low 
Reliability  Rel.    Rel. 


Moderately       Extremely 
High    High    High 
Rel .     Rel .    Rel . 


~0— .,  „^..b  ^..^.  x-^rniation  provided  in  the  accompanying  booklet 
regarding  various  sample  sizes  for  the  substantive  test,  complete  the 

fnllo„Jn^   „♦.„♦-„„ f. 


Again,  using  the  info 
regarding  various  s£ 
following  statement. 


Substantive  Test   (Sample  size  Table  3  is  derived  from  a  statistical 
variables  sampling  approach.) 

As  a  result  of  the  information  I  have  been  provided  and  the  specific 
results  of  the  compliance  tests,  I  would  recommend  as  part  of  a  final 

audit  program  the  selection  of  customers  from  the  customer 

accounts  file  as  of  11/30/78  and  that  positive  confirmations  be  sent 
to  those  selected.   (You  must  use  a  sample  size  from  Table  3.) 
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CASE  lb 

-  Assume  that  your  recommended  sample  sizes  have  been  used  in  the 
compliance  testing  of  the  internal  controls  that  were  found  to  be 
present. 

The  results  of  the  compliance  tests  can  be  summarized  as  follows: 

CT-1:  Noncompliance  rate  =  2% 

CT-2:  Noncompliance  rate  =  8% 

CT-3 :  Noncompliance  rate  =  2% 


QUESTION:   As  a  result  of  all  the  information  you  have  been  provided 

concerning  TMC  to  this  point  (including  the  specific  results 
of  your  suggested  compliance  tests  provided  above),  what 
degree  of  reliability  would  you  assign  to  the  given  controls 
with  regard  to  meeting  the  specific  objective  of  interest? 
(Circle  the  appropriate  number. ) 


Moderately 
No      Low     Low 
Reliability  Rel.     Rel. 


Moderately       Extremely 
High    High    High 
Rel.     Rel.    Rel. 


Again,  using  the  information  provided  in  the  accompanying  booklet 
regarding  various  sample  sizes  for  the  substantive  test,  complete  the 
following  statement. 


Substantive  Test 


(Sample  size  Table  3  is  derived  from  a  statistical 
variables  sampling  approach.) 


As  a  result  of  the  information  I  have  been  provided  and  the  specific 
results  of  the  compliance  tests,  I  would  recommend  as  part  of  a  final 

audit  program  the  selection  of  customers  from  the  customer 

accounts  file  as  of  11/30/78  and  that  positive  confirmations  be  sent 
to  those  selected.   (You  must  use  a  sample  size  from  Table  3.) 
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CASE  lc 

-  Assume  that  your  recommended  sample  sizes  have  been  used  in  the 
compliance  testing  of  the  internal  controls  that  were  found  to  be 
present. 

The  results  of  the  compliance  tests  can  be  summarized  as  follows: 

CT-1 :  Noncompliance  rate  =  8% 
CT-2 :  Noncompliance  rate  =  8% 
CT-3:   Noncompliance  rate  =  2% 


QUESTION:   As  a  result  of  all  the  information  you  have  been  provided 

concerning  TMC  to  this  point  (including  the  specific  results 
of  your  suggested  compliance  tests  provided  above),  what 
degree  of  reliability  would  you  assign  to  the  given  controls 
with  regard  to  meeting  the  specific  objective  of  interest? 
(Circle  the  appropriate  number. ) 


Moderately 
No      Low     Low 
Reliability  Rel.     Rel . 


Moderately       Extremely 
High    High    High 
Rel.    Rel.    Rel. 


Again,  using  the  information  provided  in  the  accompanying  booklet 
regarding  various  sample  sizes  for  the  substantive  test,  complete  the 
following  statement. 


Substantive  Test 


(Sample  size  Table  3  is  derived  from  a  statistical 
variables  sampling  approach.) 


As  a  result  of  the  information  I  have  been  provided  and  the  specific 
results  of  the  compliance  tests,  I  would  recommend  as  part  of  a  final 

audit  program  the  selection  of  customers  from  the  customer 

accounts  file  as  of  11/30/78  and  that  positive  confirmations  be  sent 
to  those  selected.   (You  must  use  a  sample  size  from  Table  3.) 
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CASE  Id 

-  Assume  that  your  recommended  sample  sizes  have  been  used  in  the 
compliance  testing  of  the  internal  controls  that  were  found  to  be 
present. 


The  results  of  the  compliance  tests  can  be  summarized  as  follows: 


CT-1 
CT-2 
CT-3 


Noncompliance  rate  =  2% 
Noncompliance  rate  =  8% 
Noncompliance  rate  =  8% 


QUESTION:   As  a  result  of  all  the  information  you  have  been  provided 

concerning  TMC  to  this  point  (including  the  specific  results 
of  your  suggested  compliance  tests  provided  above),  what 
degree  ££  reliability  would  you  assign  to  the  given  controls 
with  regard  to  meeting  the  specific  objective  of  interest? 
(Circle  the  appropriate  number. ) 


Moderately 
Low     Low 
Reliability  Rel.     Rel. 


Moderately       Extremely 
High    High    High 
Rel.    Rel.    Rel. 


Again,  using  the  information  provided  in  the  accompanying  booklet 
regarding  various  sample  sizes  for  the  substantive  test,  complete  the 
following  statement. 


Substantive  Test 


(Sample  size  Table  3  is  derived  from 
variables  sampling  approach.) 


a  statistical 


As  a  result  of  the  information  I  have  been  provided  and  the  specific 
results  of  the  compliance  tests,  I  would  recommend  as  part  of  a  final 

audit  program  the  selection  of  customers  from  the  customer 

accounts  file  as  of  11/30/78  and  that  positive  confirmations  be  sent 
to  those  selected.   (You  must  use  a  sample  size  from  Table  3.) 


156 


CASE  le 

-  Assume  that  your  recommended  sample  sizes  have  been  used  in  the 
compliance  testing  of  the  internal  controls  that  were  found  to  be 
present. 

The  results  of  the  compliance  tests  can  be  summarized  as  follows: 
CT-1 :   Noncompliance  rate  =  8% 
CT-2:   Noncompliance  rate  =  2% 
CT-3:   Noncompliance  rate  -  8% 


QUESTION:   As  a  result  of  all  the  information  you  have  been  provided 

concerning  TMC  to  this  point  (including  the  specific  results 
of  your  suggested  compliance  tests  provided  above),  what 
degree  of  reliability  would  you  assign  to  the  given  controls 
with  regard  to  meeting  the  specific  objective  of  interest? 
(Circle  the  appropriate  number. ) 


Moderately 
No      Low     Low 
Reliability  Rel .     Rel. 


Moderately       Extremely 
High    High    High 
Rel.     Rel.     Rel. 


Again,  using  the  information  provided  in  the  accompanying  booklet 
regarding  various  sample  sizes  for  the  substantive  test,  complete  the 
following  statement. 


Substantive  Test 


(Sample  size  Table  3  is  derived  from  a  statistical 
variables  sampling  approach.) 


As  a  result  of  the  information  I  have  been  provided  and  the  specific 
results  of  the  compliance  tests,  I  would  recommend  as  part  of  a  final 

audit  program  the  selection  of  customers  from  the  customer  accounts 

file  as  of  11/30/78  and  that  positive  confirmations  be  sent  to  those 
selected.   (You  must  use  a  sample  size  from  Table  3.) 
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CASE  If 

-  Assume  that  your  recommended  sample  sizes  have  been  used  in  the 
compliance  testing  of  the  internal  controls  that  were  found  to  be 
present. 

The  results  of  the  compliance  tests  can  be  summarized  as  follows: 
CT-1:   Noncompliance  rate  =  2% 
CT-2:   Noncompliance  rate  =  2% 
CT-3:   Noncompliance  rate  =  2% 


QUESTION:   As  a  result  of  all  the  information  you  have  been  provided 

concerning  TMC  to  this  point  (including  the  specific  results 
of  your  suggested  compliance  tests  provided  above),  what 
deg_ree  of  reliability  would  you  assign  to  the  given  controls 
with  regard  to  meeting  the  specific  objective  of  interest? 
(Circle  the  appropriate  number. ) 


Moderately 
No      Low     Low 
Reliability  Rel .     Rel. 


Moderately       Extremely 
High    High    High 
Rel.     Rel.     Rel. 


Again,  using  the  information  provided  in  the  accompanying  booklet 
regarding  various  sample  sizes  for  the  substantive  test,  complete  the 
following  statement. 


Substantive  Test 


(Sample  size  Table  3  is  derived  from  a  statistical 
variables  sampling  approach.) 


As  a  result  of  the  information  I  have  been  provided  and  the  specific 
results  of  the  compliance  tests,  I  would  recommend  as  part  of  a  final 

audit  program  the  selection  of  customers  from  the  customer 

accounts  file  as  of  11/30/78  and  that  positive  confirmations  be  sent  to 
those  selected.   (You  must  use  a  sample  size  from  Table  3.) 
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CASE  lg 

-  Assume  that  your  recommended  sample  sizes  have  been  used  in  the 
compliance  testing  of  the  internal  controls  that  were  found  to  be 
present. 

The  rpsults  of  the  compliance  tests  can  be  summarized  as  follows: 
CT-1:   Noncompliance  rate  =  8% 
CT-2:   Noncompliance  rate  =  8% 
CT-3:   Noncompliance  rate  =  8% 


QUESTION:   As  a  result  of  all  the  information  you  have  been  provided 

concerning  TMC  to  this  point  (including  the  specific  results 
of  your  suggested  compliance  tests  provided  above),  what 
degree  °£  reliability  would  you  assign  to  the  given  controls 
with  regard  to  meeting  the  specific  objective  of  interest? 
(Circle  the  appropriate  number. ) 


Moderately 
No      Low     Low 
Reliability  Rel.     Rel . 


Moderately       Extremely 
High    High    High 
Rel.     Rel.     Rel. 


Again,  using  the  information  provided  in  the  accompanying  booklet 
regarding  various  sample  sizes  for  the  substantive  test,  complete  the 
following  statement. 


Substantive  Test   (Sample  size  Table  3  is  derived  from  a  statistical 
variables  sampling  approach.) 

As  a  result  of  the  information  I  have  been  provided  and  the  specific 
results  of  the  compliance  tests,  I  would  recommend  as  part  of  a  final 

audit  program  the  selection  of  customers  from  the  customer 

accounts  file  as  of  11/30/78  and  that  positive  confirmations  be  sent 
to  those  selected.   (You  must  use  a  sample  size  from  Table  3.) 
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CASE  lh 

-  Assume  that  your  recommended  sample  sizes  have  been  used  in  the 
compliance  testing  of  the  internal  controls  that  were  found  to  be 
present. 

The  results  of  the  compliance  tests  can  be  summarized  as  follows: 
CT-1:   Noncompliance  rate  =  2% 
CT-2 :   Noncompliance  rate  =  2% 
CT-3:   Noncompliance  rate  =  8% 


QUESTION:   As  a  result  of  all  the  information  you  have  been  provided 

concerning  TMC  to  this  point  (including  the  specific  results 
of  your  suggested  compliance  tests  provided  above),  what 
degree  of  reliability  would  you  assign  to  the  given  controls 
with  regard  to  meeting  the  specific  objective  of  interest? 
(Circle  the  appropriate  number. ) 


Moderately 
No      Low     Low 
Reliability  Rel.     Rel. 


Moderately       Extremely 
High    High    High 
Rel .    Rel .    Rel . 


Again,  using  the  information  provided  in  the  accompanying  booklet 
regarding  various  sample  sizes  for  the  substantive  test,  complete  the 
following  statement. 


Substantive  Test   (Sample  size  Table  3  is  derived  from  a  statistical 
variables  sampling  approach.) 

As  a  result  of  the  information  I  have  been  provided  and  the  specific 
results  of  the  compliance  tests,  I  would  recommend  as  part  of  a  final 

audit  program  the  selection  of  customers  from  the  customer 

accounts  file  as  of  11/30/78  and  that  positive  confirmations  be  sent  to 
those  selected.   (You  must  use  a  sample  size  from  Table  3.) 
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-   SCENARIO   #2   - 
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SCENARIO  #2_  -    The  following  information  is  common  to  CASES  2a  through 
2d.   Please  disregard  any  information  concerning 
SCENARIO  #1. 


The  internal  control  questionnaire  shows  the  following  results  relative 
to  the  existence  of  the  key  controls  discussed  in  the  information  pro- 
vided previously. 


IC-1:  YES,  procedures  exist  to  control  for  credit  approval. 

IC"2:   N0>  procedures  do  not  exist  to  control  for  shipment  authorization. 

ICX3:  YES,  procedures  exist  to  control  for  the  physical  shipment  of  goods. 


QUESTION:   As  a  result  of  all  the  information  you  have  been  provided 

concerning  TMC  to  this  point  (including  the  specific  infor- 
mation in  this  scenario  concerning  the  existence  of  the  two 
key  controls),  what  degree  of  reliability  would  you  assign 
to  the  given  controls  with  regard  to  meeting  the  specific 
objective  of  interest?   (Circle  the  appropriate  number.) 

Moderately 
No      Low     Low 
Reliability  Rel.     Rel. 


Moderately 

Extremely 

High    High 

High 

Rel.    Rel. 

Rel. 

The  following  statements  require  your  response  as  to  the  most  appropri- 
ate sample  sizes  in  a  preliminary  audit  program  (See  NOTE  below.)  for 
both  compliance  and  substantive  procedures.   Using  the  information 
provided  in  the  accompanying  booklet  regarding  various  sample  sizes  for 
compliance  and  substantive  tests,  complete  the  following  statements. 
(You  must  select  a  sample  size  from  those  provided.) 

Compliance  Tests  -  In  the  selection  of  sample  sizes  for  the  compliance 
tests,  please  treat  each  test  individually  although  the  sampling  unit 
(sales  invoice  packet)  will  be  the  same  for  each  test  and  as  a  practical 
matter  could  be  combined. 


CT-1,  CT-2  &  CT-3: 


Sample  size  selection  and  evaluation  tables 
(Tables  1  &  2)  are  statistically  determined. 


CT~1:   (Test  of  IC-1):  I  would  recommend  that 


sales  invoices  be 


examined  to  insure  that  the  credit  approval 
procedures  have  been  followed  (indicated  by  the 
appropriate  initials  on  the  supporting  customer 
order) . 
-  This  decision  was  based  upon  a  desired  confidence  level  of       % 
(From  Table  1)  °" 
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CT-2:   (Test  of  IC-2)  :  Not  appropriate 

CT-3:   (Test  of  IC-3):  I  would  recommend  that  sales  invoices  be 

examined  to  insure  that  proper  control  procedures 
over  the  physical  shipment  of  goods  have  been 
followed  (indicated  by  proper  initials  on  the 
supporting  bill  of  lading). 

-  This  decision  was  based  upon  a  desired  confidence  level  of  %. 

(From  Table  1) 

Substantive  Test   (Sample  size  Table  3  is  statistically  derived  using  a 
variables  sampling  approach.) 

As  a  result  of  the  information  I  have  been  provided  and  in  conjunction 
with  my  planned  compliance  tests,  I  would  recommend  as  part  of  prelim- 
inary audit  program  the  selection  of  customers  from  the 

customer  accounts  file  as  of  11/30/78  and  that  positive  confirmations 
be  sent  to  those  selected.  (You  must  select  a  sample  size  from  Table 
3.) 


NOTE:   In  the  context  of  this  study,  "preliminary  audit  program"  implies 
an  initial  audit  program  that  will  possibly  be  adjusted  for  the 
substantive  test  as  a  result  of  the  evaluation  of  compliance  test 
results.   At  that  time,  a  "final  audit  program"  is  determined. 
Also,  the  substantive  test  (confirmation  of  receivables)  will  be 
performed  on  an  interim  basis  as  of  11/30/78  with  compliance 
testing  performed  for  the  period  1/1/78-11/30/78.   Appropriate 
additional  procedures  will  be  performed  for  the  month  of  Decem- 
ber. 

For  all  decisions,  please  select  the  sample  sizes  you  feel  are  most 
appropriate  prior  to  consideration  of  any  firm  restrictions  as  to  a 
minimum  sample  size  for  a  particular  compliance  or  substantive  test. 
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CASE  2a 

-  Assume  that  your  recommended  sample  sizes  have  been  used  in  the 
compliance  testing  of  the  internal  controls  that  were  found  to  be 
present. 

The  results  of  the  compliance  tests  can  be  summarized  as  follows: 

CT-1 :  Noncompliance  rate  =  2% 

CT-2:  Not  appropriate 

CT-3:  Noncompliance  rate  =  2% 


QUESTION:   As  a  result  of  all  the  information  you  have  been  provided 

concerning  TMC  to  this  point  (including  the  specific  results 
of  your  suggested  compliance  tests  provided  above) ,  what 
degree  of  reliability  would  you  assign  to  the  given  controls 
with  regard  to  meeting  the  specific  objective  of  interest? 
(Circle  the  appropriate  number . ) 

Moderately      Moderately       Extremely 
No      Low     Low  High    High    High 

Reliability  Rel.     Rel.  Rel.     Rel.     Rel. 


Again,  using  the  information  provided  in  the  accompanying  booklet 
regarding  various  sample  sizes  for  the  substantive  test,  complete  the 
following  statement. 


Substantive  Test   (Sample  size  Table  3  is  derived  from  a  statistical 
variables  sampling  approach.) 

As  a  result  of  the  information  I  have  been  provided  and  the  specific 
results  of  the  compliance  tests,  I  would  recommend  as  part  of  a  final 

audit  program  the  selection  of  customers  from  the  customer 

accounts  file  as  of  11/30/78  and  that  positive  confirmations  be  sent  to 
those  selected.   (You  must  use  a  sample  size  from  Table  3.) 
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CASE  2b 

-  Assume  that  your  recommended  sample  sizes  have  been  used  in  the 
compliance  testing  of  the  internal  controls  that  were  found  to  be 
present. 

The  results  of  the  compliance  tests  can  be  summarized  as  follows: 
CT-1:   Noncompliance  rate  =  8% 
CT-2:   Not  appropriate 
CT-3:   Noncompliance  rate  =  2% 


QUESTION:   As  a  result  of  all  the  information  you  have  been  provided 

concerning  TMC  to  this  point  (including  the  specific  results 
of  your  suggested  compliance  tests  provided  above),  what 
dje£r_ee  of  reliability  would  you  assign  to  the  given  controls 
with  regard  to  meeting  the  specific  objective  of  interest? 
(Circle  the  appropriate  number . ) 

Moderately      Moderately       Extremely 
No      Low     Low  High    High    High 

Reliability  Rel.     Rel .  Rel.     Rel.     Rel. 


Again,  using  the  information  provided  in  the  accompanying  booklet 
regarding  various  sample  sizes  for  the  substantive  test,  complete  the 
following  statement. 


Substantive  Test   (Sample  size  Table  3  is  derived  from  a  statistical 
variables  sampling  approach.) 

As  a  result  of  the  information  I  have  been  provided  and  the  specific 
results  of  the  compliance  tests,  I  would  recommend  as  part  of  a  final 

audit  program  the  selection  of  customers  from  the  customer 

accounts  file  as  of  11/30/78  and  that  positive  confirmations  be  sent  to 
those  selected.   (You  must  use  a  sample  size  from  Table  3.) 


165 


CASE  2c 

-  Assume  that  your  recommended  sample  sizes  have  been  used  in  the 
compliance  testing  of  the  internal  controls  that  were  found  to  be 
present. 

The  results  of  the  compliance  tests  can  be  summarized  as  follows: 
CT-1:   Noncompliance  rate  =  8% 
CT-2:   Not  appropriate 
CT-3:   Noncompliance  rate  =  8% 


QUESTION:   As  a  result  of  all  the  information  you  have  been  provided 

concerning  TMC  to  this  point  (including  the  specific  results 
of  your  suggested  compliance  tests  provided  above),  what 
degree  of  reliability  would  you  assign  to  the  given  controls 
with  regard  to  meeting  the  specific  objective  of  interest?) 
(Circle  the  appropriate  number . ) 


Moderately 
No      Low     Low 
Reliability  Rel.     Rel. 


Moderately       Extremely 
High    High    High 
Rel.     Rel.    Rel. 


Again,  using  the  information  provided  in  the  accompanying  booklet 
regarding  various  sample  sizes  for  the  substantive  test,  complete  the 
following  statement. 


Substantive  Test 


(Sample  size  Table  3  is  derived  from  a  statistical 
variables  sampling  approach. ) 


As  a  result  of  the  information  I  have  been  provided  and  the  specific 
results  of  the  compliance  tests,  I  would  recommend  as  part  of  a  final 

audit  program  the  selection  of  customers  from  the  customer 

accounts  file  as  of  11/30/78  and  that  positive  confirmations  be  sent  to 
those  selected.   (You  must  use  a  sample  size  from  Table  3.) 
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CASE  2d 

-  Assume  that  your  recommended  sample  sizes  have  been  used  in  the 
compliance  testing  of  the  internal  controls  that  were  found  to  be 
present. 

The  results  of  the  compliance  tests  can  be  summarized  as  follows: 
CT-1:   Noncompliance  rate  =  2% 
CT-2:   Not  appropriate 
CT-3:   Noncompliance  rate  =  8% 


QUESTION:   As  a  result  of  all  the  information  you  have  been  provided 

concerning  TMC  to  this  point  (including  the  specific  results 
of  your  suggested  compliance  tests  provided  above),  what 
degree  of  reliability  would  you  assign  to  the  given  controls 
with  regard  to  meeting  the  specific  objective  of  interest? 
(Circle  the  appropriate  number . ) 


Moderately 
No      Low     Low 
Reliability  Rel.     Rel. 


Moderately       Extremely 
High    High    High 
Rel.     Rel.     Rel. 


Again,  using  the  information  provided  in  the  accompanying  booklet 
regarding  various  sample  sizes  for  the  substantive  test,  complete  the 
following  statement. 


Substantive  Test 


(Sample  size  Table  3  is  derived  from  a  statistical 
variables  sampling  approach.) 


As  a  result  of  the  information  I  have  been  provided  and  the  specific 
results  of  the  compliance  tests,  I  would  recommend  as  part  of  a  final 

audit  program  the  selection  of  customers  from  the  customer 

accounts  file  as  of  11/30/78  and  that  positive  confirmations  be  sent  to 
those  selected.   (You  must  use  a  sample  size  from  Table  3.) 


167 


-  STATISTICAL  SAMPLING  TABLES  - 

TABLE  1:   Sample  Sizes  for  CT-1  and  CT-2 

TABLE  2:   Evaluation  of  Compliance  Test  Results 

TABLE  3:   Sample  Sizes  for  Substantive  Test 
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SAMPLE  SIZES  FOR  SUBSTANTIVE  TEST 

(You  must  decide  on  a  sample  size  based  upon  your  choice  for  the  values 
of  Z  and  M. )* 

TABLE  3 


M  CL 

(Z=1.65) 

90% 

(Z=1.96) 

95% 

(Z=2.24) 
97.5% 

(Z=2.58) 
99% 

10,000 

390 

550 

720 

959 

15,000 

175 

245 

320 

426 

20,000 

100 

140 

180 

240 

25,000 

65 

90 

115 

153 

30,000 

43 

62 

80 

107 

35,000 

30 

44 

60 

78 

40,000 

25 

35 

45 

61 

45,000 

20 

27 

36 

47 

50,000 

15 

22 

29 

38 

*  The  table  was  developed  from  the  application  of  a  difference  estimation 
approach.   Assume  any  assumptions  necessary  for  the  use  of  the  differ- 
ence estimation  approach  are  met  and  and  table  above  is  derived  from  the 
following  equation: 

n-z-sdI  2 


M 
where, 

n  =  estimated  sample  size 

N  =  population  size  (Fixed  at  6,000) 

Z  =  confidence  coefficient  for  various  confidence  levels  (CL) 
Z  =  1.65  for  90%  CL 
Z  =  1.96  for  95%  CL 
Z  =  2.24  for  97.5%  CL 
Z  =  2.58  for  99%  CL 

SD  =  estimated  standard  deviation  of  population  differences 

(Fixed  at  $20,  as  estimated  through  appropriate  inquiry.) 

M  =  amount  judged  material  for  the  Sales-Collection  Cycle 
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POSTEXPERIMENT  QUESTIONNAIRE 

Now  that  you  have  completed  the  requirements  for  the  12  Cases,  please 
allocate  100  points  to  the  three  key  internal  controls  in  such  a  way 
as  to  indicate  the  relative  importance  of  each  in  making  your  deci- 
sions.  The  more  important  control  should  be  assigned  more  points  than 
the  less  important  control,  and  the  total  points  should  equal  100. 

_______  points   IC-1   (Credit  approval) 

points   IC-2   (Shipment  authorization) 

______  points   IC-3   (Physical  shipment) 


100   TOTAL  POINTS 


2.   Indicate  how  realistic  or  representative  this  task  was  in  comparison 
to  actual  audit  judgments  of  a  similar  nature.   (Circle  the  appropri- 
ate number. ) 

Highly  Not        Moderately  Very 

Unrepresentative  Representative  Representative  Representative  Representative 


3.   Indicate  what  you  feel  is  your  degree  of  familiarity  with  statistical 
sampling.   (Circle  the  appropriate  number.) 

Very  Moderately      Not  Very        Highly 

Familiar        Familiar       Familiar       Familiar  Unfamiliar 


4.   What  other  information  would  you  consider  to  be  important  in  making 

similar  types  of  judgment  decisions  that  was  not  included  in  this  study? 


5.  Please  indicate  the  extent  of  your  accounting  experience. 

Public  Accounting  years;  Industry  years;  Other  years. 

Current  Position  in  Firm 
Age  

6.  Please  indicate  if  you  are  interested  in  receiving  the  results  of  this 
study. 

NO  YES,  with  names  of  individuals  and/or  firms  not  revealed. 

If  YES,  please  indicate:   NAME  

ADDRESS 
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Please  respond  to  the  following  questions  by  circling  the  appropriate 
response.   This  is  not  a  test;  there  are  no  right  or  wrong  answers.   It 
is  important  that  you  try  to  answer  each  question  honestly. 


Suppose  this  same  experiment  was  to  be  repeated  on  a  number  of  future 
occasions.   How  many  more  times  would  you  be  prepared  to  participate 
in  the  experiment  under  circumstances  similar  to  the  present? 

a****** ********* a ***** ********************************* 

******* 

No       1        2        3        4        5      More 

More     More     More     More     More     More    Than  5 

Times     Time     Times    Times    Times    Times    Times 


2.   Did  your  desire  to  perform  well  in  undertaking  a  challenging  task 
cause  you  to  try  very  hard? 

******************************************************* 

******* 

Definitely   Yes    Probably   Don't   Probably    No   Definitely 

Yes  Yes      Know      No  No 


3.   Did  you  enjoy  making  the  decisions  required  in  the  experiment? 

******************************************************* 
******* 

Definitely   No    Probably   Don't   Probably    Yes   Definitely 
No  No      Know      Yes  Yes 


4.   Are  you  satisfied  with  your  performance  in  the  experiment? 

******************************************************* 

******* 

Definitely   Yes    Probably   Don't   Probably    No   Definitely 

Yes  Yes      Know      No  No 


5.   Did  you  feel  tense  during  the  experiment? 

******************************************************* 

******* 

Definitely   Yes    Probably   Don't   Probably    No   Definitely 

Yes  Yes      Know      No  No 


173 


6.   Did  your  desire  to  cooperate  with  the  experimenter  cause  you  to  try 
very  hard? 

******************************************************* 
******* 

Definitely   No    Probably   Don't   Probably    Yes   Definitely 
No  No      Know     Yes  Yes 


7.   Did  you  enjoy  the  overall  experience  of  participating  in  the  experi- 
ment? 

******************************************************* 
******* 

Definitely   No    Probably   Don't   Probably    Yes   Definitely 
No  No      Know     Yes  Yes 


Did  your  desire  to  contribute  to  research  knowledge  cause  you  to  try 
very  hard? 

*************************** ** **************** *** ******* 

******* 

Definitely   Yes    Probably   Don't   Probably    No   Definitely 

Yes  Yes      Know      No  No 


9.   Indicate  the  extent  to  which  you  make  decisions  that  are  similar  to 
those  included  in  this  experiment.   (Circle  the  appropriate  number.) 

Very  Very 

Seldom       Seldom      Sometimes      Often        Often 


10.   Please  comment  below  on  any  adjustments  in  your  responses  to  sample 
size  questions  had  you  applied  any  firm  guidelines  as  to  a  minimum 
sample  size. 

Compliance  Tests: 


Substantive  Tests: 
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ville,  Tennessee. 

Mr.  Tabor  received  a  Bachelor  of  Science  in  Business  Administra- 
tion majoring  in  accounting  in  August  1973  from  the  University  of  Tennes- 
see. In  August  1974  he  received  a  Master  of  Business  Administration 
majoring  in  accounting,  also  from  the  University  of  Tennessee.   From 
1974  until  1976  Mr.  Tabor  was  a  Visiting  Instructor  of  Accounting  at  the 
University  of  Evansville,  Evansville,  Indiana.   From  June  1976  until  the 
present  Mr.  Tabor  has  attended  the  University  of  Florida  where  he  has 
studied  for  a  Doctor  of  Philosophy  degree  in  business  administration 
(accounting) . 
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I  certify  that  I  have  read  this  study  and  that  in  my  opinion  it 
conforms  to  acceptable  standards  of  scholarly  presentation  and  is  fully 
adequate,  in  scope  and  quality,  as  a  dissertation  for  the  degree  of 
Doctor  of  Philosophy. 


E.  Dan  Smith,  Chairman 

Associate  Professor  of  Accounting 


I  certify  that  I  have  read  this  study  and  that  in  my  opinion  it 

conforms  to  acceptable  standards  of  scholarly  presentation  and  is  fully 

adequate,  in  scope  and  quality,  as  a  dissertation  for  the  degree  of 
Doctor  of  Philosophy. 


Vo\Al&4   A.  T.  Snowball 

Associate  Professor  of  Accounting 


I  certify  that  I  have  read  this  study  and  that  in  my  opinion  it 

conforms  to  acceptable  standards  of  scholarly  presentation  and  is  fully 

adequate,  in  scope  and  quality,  as  a  dissertation  for  the  degree  of 
Doctor  of  Philosophy. 


Assistant  Professor  of  Accounting 


I  certify  that  I  have  read  this  study  and  that  in  my  opinion  it 
conforms  to  acceptable  standards  of  scholarly  presentation  and  is  fully 
adequate,  in  scope  and  quality,  as  a  dissertation  for  the  degree  of 
Doctor  of  Philosophy. 


Harry  J , 

Professor  of  Management 


This  dissertation  was  submitted  to  the  Graduate  Faculty  of  the  School 
of  Accounting  in  the  College  of  Business  Administration  and  to  the 
Graduate  Council,  and  was  accepted  as  partial  fulfillment  of  the 
requirements  for  the  degree  of  Doctor  of  Philosophy. 


Dean,  Graduate  School 
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